mlsanityData to 7 Oct 2026

Neural Arithmetic Logic Units

arXiv 1808.00508 ↗First posted 1 Aug 2018cs.NEcited by 69

Not on the map. Why?

Abstract (arXiv)

Neural networks can learn to represent and manipulate numerical information, but they seldom generalize well outside of the range of numerical values encountered during training. To encourage more systematic numerical extrapolation, we propose an architecture that represents numerical quantities as linear activations which are manipulated using primitive arithmetic operators, controlled by learned gates. We call this module a neural arithmetic logic unit (NALU), by analogy to the arithmetic logic unit in traditional processors. Experiments show that NALU-enhanced neural networks can learn to track time, perform arithmetic over images of numbers, translate numerical language into real-valued scalars, execute computer code, and count objects in images. In contrast to conventional architectures, we obtain substantially better generalization both inside and outside of the range of numerical values encountered during training, often extrapolating orders of magnitude beyond trained numerical ranges.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 19; 2022: 14; 2023: 14; 2024: 4; 2025: 5; 2026: 12.

69 papers.

Sort:

By area:

  1. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  2. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  3. 2021Combinatorial optimization and reasoning with graph neural networkscited by 267
  4. 2021Investigating the Limitations of Transformers with Simple Arithmetic Taskscited by 136
  5. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 103
  6. 2021Representing Numbers in NLP: a Survey and a Visioncited by 83
  7. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  8. 2023Length Generalization in Arithmetic Transformerscited by 64
  9. 2021Linear algebra with transformerscited by 62
  10. 2022Theory of Graph Neural Networks: Representation and Learningcited by 60
  11. 2022Deep Symbolic Regression for Recurrent Sequencescited by 53
  12. 2021Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEscited by 42
38 more
  1. 2022Relational Attention: Generalizing Transformers for Graph-Structured Taskscited by 41
  2. 2023Learning the greatest common divisor: explaining transformer predictionscited by 33
  3. 2021Dangers of Bayesian Model Averaging under Covariate Shiftcited by 31
  4. 2022SALSA: Attacking Lattice Cryptography with Transformerscited by 28
  5. 2021Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematicscited by 27
  6. 2022Deep Learning and Symbolic Regression for Discovering Parametric Equationscited by 22
  7. 2023Generative learning for nonlinear dynamicscited by 22
  8. 2023Can neural networks do arithmetic? A survey on the elementary numerical skills of state-of-the-art deep learning modelscited by 18
  9. 2021MC-LSTM: Mass-Conserving LSTMcited by 17
  10. 2021Pretrained Language Models are Symbolic Mathematics Solvers too!cited by 17
  11. 2023Neural Algorithmic Reasoning Without Intermediate Supervisioncited by 16
  12. 2024Softened Symbol Grounding for Neuro-symbolic Systemscited by 16
  13. 2021Investigating Numeracy Learning Ability of a Text-to-Text Transfer Modelcited by 13
  14. 2021Truth-Conditional Captioning of Time Series Datacited by 11
  15. 2022Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Modelscited by 10
  16. 2024Arithmetic Feature Interaction Is Necessary for Deep Tabular Learningcited by 8
  17. 2022Transformers discover an elementary calculation system exploiting local attention and grid-like problem representationcited by 7
  18. 2022Automatic Rule Induction for Interpretable Semi-Supervised Learningcited by 6
  19. 2023Deciphering Raw Data in Neuro-Symbolic Learning with Provable Guaranteescited by 6
  20. 2023A Comprehensive Evaluation of Tool-Assisted Generation Strategiescited by 6
  21. 2025KANO: Kolmogorov-Arnold Neural Operatorcited by 6
  22. 2021Neural Sequence-to-grid Module for Learning Symbolic Rulescited by 5
  23. 2021Evolutionary Training and Abstraction Yields Algorithmic Generalization of Neural Computerscited by 5
  24. 2021Learning to Pool in Graph Neural Networks for Extrapolationcited by 4
  25. 2022Asynchronous Neural Networks for Learning in Graphscited by 4
  26. 2023Exploring the Approximation Capabilities of Multiplicative Neural Networks for Smooth Functionscited by 4
  27. 2025Learning to Add, Multiply, and Execute Algorithmic Instructions Exactly with Neural Networkscited by 4
  28. 2022Logical Tasks for Measuring Extrapolation and Rule Comprehensioncited by 3
  29. 2024Acceleration of Grokking in Learning Arithmetic Operations via Kolmogorov-Arnold Representationcited by 3
  30. 2025Teaching Metric Distance to Discrete Autoregressive Language Modelscited by 3
  31. 2022Physics Informed Symbolic Networkscited by 2
  32. 2023Towards a Neural Lambda Calculus: Neurosymbolic AI Applied to the Foundations of Functional Programmingcited by 2
  33. 2023NeuralFastLAS: Fast Logic-Based Learning from Raw Datacited by 2
  34. 2026Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectorscited by 2
  35. 2021Multi-Class Multi-Instance Count Conditioned Adversarial Image Generationcited by 1
  36. 2022Open World Learning Graph Convolution for Latency Estimation in Routing Networkscited by 1
  37. 2023Learning to solve arithmetic problems with a virtual abacuscited by 1
  38. 2023A Hybrid System for Systematic Generalization in Simple Arithmetic Problemscited by 1

50 most cited of 69 All 69 (JSON)