Accuracy and Architecture Studies of Residual Neural Network solving Ordinary Di

Accuracy and Architecture Studies of Residual Neural Network solving Ordinary Differential Equations

In this paper we consider utilizing a residual neural network (ResNet) to solve ordinary differential equations. Stochastic gradient descent method is applied to obtain the optimal parameter set of weights and biases of the network.We apply forward Euler, Runge-Kutta2 and Runge-Kutta4 finite difference methods to generate three sets of targets training the ResNet and carry out the target study. The well trained ResNet behaves just as its counterpart of the corresponding one-step finite difference method. In particular, we carry out (1) the architecture study in terms of number of hidden layers and neurons per layer to find the optimal ResNet structure; (2) the target study to verify the ResNet solver behaves as accurate as its finite difference method counterpart; (3) solution trajectory simulation. Even the ResNet solver looks like and is implemented in a way similar to forward Euler scheme, its accuracy can be as high as any one step method. A sequence of numerical examples are presented to demonstrate the performance of the ResNet solver.

残差神经网络求解常微分方程的精度和体系结构研究

在本文中，我们考虑利用残差神经网络（ResNet）求解常微分方程。应用随机梯度下降法获得网络权重和偏差的最优参数集。.. 我们应用正向Euler，Runge-Kutta2和Runge-Kutta4有限差分方法来生成三组训练ResNet的目标并进行目标研究。训练有素的ResNet的行为与相应的一步有限差分法的行为相同。特别地，我们进行（1）关于隐藏层数和每层神经元的体系结构研究，以找到最佳的ResNet结构；（2）验证ResNet解算器的行为与其有限差分法对应物一样准确的目标研究；（3）求解轨迹仿真。即使是ResNet求解器，其外观和实现方式与正向Euler方案相似，其准确性也可以高达任何一步法。给出了一系列数值示例，以演示ResNet求解器的性能。 （阅读更多）