探索卷积神经网络的未探索张量网络分解

张量分解方法被广泛用于卷积神经网络（CNN）中的模型压缩和快速推断。尽管可以想到许多分解方法，但实际上仅使用了CP分解方法，还进行了其他一些分解，并且在可用方法之间未进行广泛的比较。..

Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks

Tensor decomposition methods are widely used for model compression and fast inference in convolutional neural networks (CNNs). Although many decompositions are conceivable, only CP decomposition and a few others have been applied in practice, and no extensive comparisons have been made between available methods.Previous studies have not determined how many decompositions are available, nor which of them is optimal. In this study, we first characterize a decomposition class specific to CNNs by adopting a flexible graphical notation. The class includes such well-known CNN modules as depthwise separable convolution layers and bottleneck layers, but also previously unknown modules with nonlinear activations. We also experimentally compare the tradeoff between prediction accuracy and time/space complexity for modules found by enumerating all possible decompositions, or by using a neural architecture search. We find some nonlinear decompositions outperform existing ones.