# A Primer for Neural Arithmetic Logic Modules

arXiv: https://arxiv.org/abs/2101.09530 · HTML: https://mlsanity.com/paper/2101.09530/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2021.

## Built on by (most cited first, 7 of 7)

- [Investigating Symbolic Capabilities of Large Language Models](https://mlsanity.com/paper/2405.13209/) (2024)
- [Learning to Add, Multiply, and Execute Algorithmic Instructions Exactly with Neural Networks](https://mlsanity.com/paper/2502.16763/) (2025)
- [Learning Division with Neural Arithmetic Logic Modules](https://mlsanity.com/paper/2110.05177/) (2021)
- [Improving the Robustness of Neural Multiplication Units with Reversible Stochasticity](https://mlsanity.com/paper/2211.05624/) (2022)
- [The Domain Mixed Unit: A New Neural Arithmetic Layer](https://mlsanity.com/paper/2509.08180/) (2025)
- [BRo-JEPA: Learning Modular Transformations in Latent Space](https://mlsanity.com/paper/2606.01372/) (2026)
- [TriPLU: Bypassing the Gate with Direct Trilinear Product FFNs in Tiny Language Models](https://mlsanity.com/paper/2608.20360/) (2026)

## Abstract (arXiv)

Neural Arithmetic Logic Modules have become a growing area of interest, though remain a niche field. These modules are neural networks which aim to achieve systematic generalisation in learning arithmetic and/or logic operations such as +, -, ×, div, ≤, textrmAND while also being interpretable. This paper is the first in discussing the current state of progress of this field, explaining key works, starting with the Neural Arithmetic Logic Unit (NALU). Focusing on the shortcomings of the NALU, we provide an in-depth analysis to reason about design choices of recent modules. A cross-comparison between modules is made on experiment setups and findings, where we highlight inconsistencies in a fundamental experiment causing the inability to directly compare across papers. To alleviate the existing inconsistencies, we create a benchmark which compares all existing arithmetic NALMs. We finish by providing a novel discussion of existing applications for NALU and research directions requiring further exploration.