# Neural Programmer: Inducing Latent Programs with Gradient Descent

arXiv: https://arxiv.org/abs/1511.04834 · HTML: https://mlsanity.com/paper/1511.04834/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2015.

## Built on by (most cited first, 30 of 34)

- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [TAPEX: Table Pre-training via Learning a Neural SQL Executor](https://mlsanity.com/paper/2107.07653/) (2021)
- [Binding Language Models in Symbolic Languages](https://mlsanity.com/paper/2210.02875/) (2022)
- [Neural Production Systems: Learning Rule-Governed Visual Dynamics](https://mlsanity.com/paper/2103.01937/) (2021)
- [Open Domain Question Answering over Tables via Dense Retrieval](https://mlsanity.com/paper/2103.12011/) (2021)
- [Learning to Synthesize Programs as Interpretable and Generalizable Policies](https://mlsanity.com/paper/2108.13643/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [A Survey on Table Question Answering: Recent Advances](https://mlsanity.com/paper/2207.05270/) (2022)
- [Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering](https://mlsanity.com/paper/2108.02866/) (2021)
- [UQE: A Query Engine for Unstructured Databases](https://mlsanity.com/paper/2407.09522/) (2024)
- [PyGlove: Symbolic Programming for Automated Machine Learning](https://mlsanity.com/paper/2101.08809/) (2021)
- [From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning](https://mlsanity.com/paper/2304.07995/) (2023)
- [Generate, Transform, Answer: Question Specific Tool Synthesis for Tabular Data](https://mlsanity.com/paper/2303.10138/) (2023)
- [Hierarchical Neural Program Synthesis](https://mlsanity.com/paper/2303.06018/) (2023)
- [Evolutionary Training and Abstraction Yields Algorithmic Generalization of Neural Computers](https://mlsanity.com/paper/2105.07957/) (2021)
- [TableQuery: Querying tabular data with natural language](https://mlsanity.com/paper/2202.00454/) (2022)
- [Learning Program Behavioral Models from Synthesized Input-Output Pairs](https://mlsanity.com/paper/2407.08597/) (2024)
- [Learning to Add, Multiply, and Execute Algorithmic Instructions Exactly with Neural Networks](https://mlsanity.com/paper/2502.16763/) (2025)
- [Gradient-Based Program Repair: Fixing Bugs in Continuous Program Spaces](https://mlsanity.com/paper/2505.17703/) (2025)
- [Probabilistic Graph Reasoning for Natural Proof Generation](https://mlsanity.com/paper/2107.02418/) (2021)
- [Weakly Supervised Reasoning by Neuro-Symbolic Approaches](https://mlsanity.com/paper/2309.13072/) (2023)
- [A Comprehensive Review of State-of-The-Art Methods for Java Code Generation from Natural Language Text](https://mlsanity.com/paper/2306.06371/) (2023)
- [Exploring Language Model's Code Generation Ability with Auxiliary Functions](https://mlsanity.com/paper/2403.10575/) (2024)
- [PuzzleGPT: Emulating Human Puzzle-Solving Ability for Time and Location Prediction](https://mlsanity.com/paper/2501.14210/) (2025)
- [Generalizable Reasoning through Compositional Energy Minimization](https://mlsanity.com/paper/2510.20607/) (2025)
- [Geometry of Program Synthesis](https://mlsanity.com/paper/2103.16080/) (2021)
- [Latent Programming Horizons in Coding Agents](https://mlsanity.com/paper/2607.05188/) (2026)
- [Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning in Robotics](https://mlsanity.com/paper/2201.10266/) (2022)
- [SMARAGD: Learning SMatch for Accurate and Rapid Approximate Graph Distance](https://mlsanity.com/paper/2203.13226/) (2022)

## Abstract (arXiv)

Deep neural networks have achieved impressive supervised classification performance in many tasks including image recognition, speech recognition, and sequence to sequence learning. However, this success has not been translated to applications like question answering that may involve complex arithmetic and logic reasoning. A major limitation of these models is in their inability to learn even simple arithmetic and logic operations. For example, it has been shown that neural networks fail to learn to add two binary numbers reliably. In this work, we propose Neural Programmer, an end-to-end differentiable neural network augmented with a small set of basic arithmetic and logic operations. Neural Programmer can call these augmented operations over several steps, thereby inducing compositional programs that are more complex than the built-in operations. The model learns from a weak supervision signal which is the result of execution of the correct program, hence it does not require expensive annotation of the correct program itself. The decisions of what operations to call, and what data segments to apply to are inferred by Neural Programmer. Such decisions, during training, are done in a differentiable fashion so that the entire network can be trained jointly by gradient descent. We find that training the model is difficult, but it can be greatly improved by adding random noise to the gradient. On a fairly complex synthetic table-comprehension dataset, traditional recurrent networks and attentional models perform poorly while Neural Programmer typically obtains nearly perfect accuracy.