Neural Sequence-to-grid Module for Learning Symbolic Rules

Neural Sequence-to-grid Module for Learning Symbolic Rules

Logical reasoning tasks over symbols, such as learning arithmetic operations and computer program evaluations, have become challenges to deep learning. In particular, even state-of-the-art neural networks fail to achieve \textit{out-of-distribution} (OOD) generalization of symbolic reasoning tasks, whereas humans can easily extend learned symbolic rules.To resolve this difficulty, we propose a neural sequence-to-grid (seq2grid) module, an input preprocessor that automatically segments and aligns an input sequence into a grid. As our module outputs a grid via a novel differentiable mapping, any neural network structure taking a grid input, such as ResNet or TextCNN, can be jointly trained with our module in an end-to-end fashion. Extensive experiments show that neural networks having our module as an input preprocessor achieve OOD generalization on various arithmetic and algorithmic problems including number sequence prediction problems, algebraic word problems, and computer program evaluation problems while other state-of-the-art sequence transduction models cannot. Moreover, we verify that our module enhances TextCNN to solve the bAbI QA tasks without external memory.

用于学习符号规则的神经序列到网格模块

符号上的逻辑推理任务（例如学习算术运算和计算机程序评估）已成为深度学习的挑战。特别是，即使是最先进的神经网络也无法实现符号推理任务的\ textit {out-of-distribution}（OOD）泛化，而人类却可以轻松地扩展学习到的符号规则。.. 为了解决这个难题，我们提出了一个神经序列到网格（seq2grid）模块，这是一种输入预处理器，可以自动将输入序列分段并将其对齐到网格中。当我们的模块通过新颖的可微分映射输出网格时，任何采用网格输入的神经网络结构（例如ResNet或TextCNN）都可以与我们的模块以端到端的方式共同训练。大量的实验表明，以我们的模块作为输入预处理器的神经网络可以对各种算术和算法问题（包括数字序列预测问题，代数词问题和计算机程序评估问题）进行OOD泛化，而其他最新的序列转换模型则无法实现。此外，我们验证了我们的模块增强了TextCNN来解决bAbI QA任务而无需外部存储器。 （阅读更多）