mlsanityData to 7 Oct 2026

Learning to Execute

arXiv 1410.4615 ↗First posted 17 Oct 2014cs.NE cs.AI cs.LGcited by 105

Not on the map. Why?

Abstract (arXiv)

Recurrent Neural Networks (RNNs) with Long Short-Term Memory units (LSTM) are widely used because they are expressive and are easy to train. Our interest lies in empirically evaluating the expressiveness and the learnability of LSTMs in the sequence-to-sequence regime by training them to evaluate short computer programs, a domain that has traditionally been seen as too complex for neural networks. We consider a simple class of programs that can be evaluated with a single left-to-right pass using constant memory. Our main result is that LSTMs can learn to map the character-level representations of such programs to their correct outputs. Notably, it was necessary to use curriculum learning, and while conventional curriculum learning proved ineffective, we developed a new variant of curriculum learning that improved our networks' performance in all experimental conditions. The improved curriculum had a dramatic impact on an addition problem, making it possible to train an LSTM to add two 9-digit numbers with 99% accuracy.

Key facts. Cited by 105 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Oct 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 23; 2022: 16; 2023: 22; 2024: 15; 2025: 17; 2026: 11.

105 papers.

Sort:

By area:

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  6. 2021Combinatorial optimization and reasoning with graph neural networkscited by 267
  7. 2021Curriculum Learning: A Surveycited by 257
  8. 2023Teaching Arithmetic to Small Transformerscited by 121
  9. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 103
  10. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
  11. 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
  12. 2023Generalization on the Unseen, Logic Reasoning and Degree Curriculumcited by 73
38 more
  1. 2024Understanding Transformer Reasoning Capabilities via Graph Algorithmscited by 64
  2. 2024NExT: Teaching Large Language Models to Reason about Code Executioncited by 60
  3. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 54
  4. 2022Relational Attention: Generalizing Transformers for Graph-Structured Taskscited by 41
  5. 2025Principled Data Selection for Alignment: The Hidden Risks of Difficult Examplescited by 37
  6. 2021Latent Execution for Neural Program Synthesiscited by 35
  7. 2024How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpadcited by 35
  8. 2022Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computingcited by 34
  9. 2024LeanAgent: Lifelong Learning for Formal Theorem Provingcited by 34
  10. 2023Code Execution with Pre-trained Language Modelscited by 33
  11. 2021Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematicscited by 27
  12. 2021How to transfer algorithmic reasoning knowledge to learn new algorithms?cited by 22
  13. 2023Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputscited by 22
  14. 2021Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absentcited by 21
  15. 2023A Mathematical Model for Curriculum Learning for Paritiescited by 19
  16. 2023Proximal Curriculum for Reinforcement Learning Agentscited by 19
  17. 2021Pretrained Language Models are Symbolic Mathematics Solvers too!cited by 17
  18. 2023ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesiscited by 17
  19. 2022Neural Approximation of Graph Topological Featurescited by 16
  20. 2023Neural Algorithmic Reasoning Without Intermediate Supervisioncited by 16
  21. 2021MLBiNet: A Cross-Sentence Collective Event Detection Networkcited by 15
  22. 2022Static Prediction of Runtime Errors by Learning to Execute Programs with External Resource Descriptionscited by 11
  23. 2023Generate, Transform, Answer: Question Specific Tool Synthesis for Tabular Datacited by 11
  24. 2025What I cannot execute, I do not understand: Training and Evaluating LLMs on Program Execution Tracescited by 11
  25. 2022Robust Reinforcement Learning via Genetic Curriculumcited by 10
  26. 2021Curriculum Design for Teaching via Demonstrations: Theory and Applicationscited by 9
  27. 2021Programming Knowledge Tracing: A Comprehensive Dataset and A New Modelcited by 9
  28. 2023RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learningcited by 8
  29. 2024Discrete Neural Algorithmic Reasoningcited by 8
  30. 2023LExecutor: Learning-Guided Executioncited by 7
  31. 2023SIP: Injecting a Structural Inductive Bias into a Seq2Seq Model by Simulationcited by 7
  32. 2022Compositional Generalization and Decomposition in Neural Program Synthesiscited by 6
  33. 2023Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimizationcited by 6
  34. 2023End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Controlcited by 6
  35. 2021Neural Sequence-to-grid Module for Learning Symbolic Rulescited by 5
  36. 2022A Neural Network Architecture for Program Understanding Inspired by Human Behaviorscited by 5
  37. 2022CRISP: Curriculum based Sequential Neural Decoders for Polar Code Familycited by 5
  38. 2024Progressive Growing of Patch Size: Resource-Efficient Curriculum Learning for Dense Prediction Taskscited by 5

50 most cited of 105 All 105 (JSON)