Stable ResNet

Stable ResNet

Deep ResNet architectures have achieved state of the art performance on many tasks. While they solve the problem of gradient vanishing, they might suffer from gradient exploding as the depth becomes large (Yang et al. 2017).Moreover, recent results have shown that ResNet might lose expressivity as the depth goes to infinity (Yang et al. 2017, Hayou et al. 2019). To resolve these issues, we introduce a new class of ResNet architectures, called Stable ResNet, that have the property of stabilizing the gradient while ensuring expressivity in the infinite depth limit.

稳定的ResNet

深度ResNet架构在许多任务上都达到了最先进的性能。尽管它们解决了梯度消失的问题，但随着深度变大，它们可能会遭受梯度爆炸的影响（Yang等人2017）。.. 此外，最近的结果表明，随着深度达到无穷远，ResNet可能会失去表达能力（Yang等人，2017； Hayou等人，2019）。为了解决这些问题，我们引入了一种称为Resable ResNet的新型ResNet架构，该架构具有稳定渐变的特性，同时确保在无限深度限制内的表现力。 （阅读更多）