Deep Neural Network Training without Multiplications

Deep Neural Network Training without Multiplications

Is multiplication really necessary for deep neural networks? Here we propose just adding two IEEE754 floating-point numbers with an integer-add instruction in place of a floating-point multiplication instruction.We show that ResNet can be trained using this operation with competitive classification accuracy. Our proposal did not require any methods to solve instability and decrease in accuracy, which is common in low-precision training. In some settings, we may obtain equal accuracy to the baseline FP32 result. This method will enable eliminating the multiplications in deep neural-network training and inference.

无需乘法的深度神经网络训练

深度神经网络真的需要乘法吗？在这里，我们建议仅使用整数加法指令代替浮点乘法指令，将两个IEEE754浮点数相加。.. 我们证明，可以使用此操作以具有竞争力的分类准确性来训练ResNet。我们的建议不需要任何解决不稳定和精度降低的方法，这在低精度训练中很常见。在某些情况下，我们可以获得与基准FP32结果相同的准确性。这种方法将能够消除深度神经网络训练和推理中的乘法。 （阅读更多）