mlsanityData to 7 Oct 2026

Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets

arXiv 1503.01007 ↗First posted 3 Mar 2015cs.NE cs.LGcited by 76

Not on the map. Why?

Abstract (arXiv)

Despite the recent achievements in machine learning, we are still very far from achieving real artificial intelligence. In this paper, we discuss the limitations of standard deep learning approaches and show that some of these limitations can be overcome by learning how to grow the complexity of a model in a structured way. Specifically, we study the simplest sequence prediction problems that are beyond the scope of what is learnable with standard recurrent networks, algorithmically generated sequences which can only be learned by models which have the capacity to count and to memorize sequences. We show that some basic algorithms can be learned from sequential data using a recurrent network associated with a trainable memory.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Mar 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 2; 2021: 19; 2022: 12; 2023: 18; 2024: 13; 2025: 9; 2026: 2.

76 papers.

Sort:

By area:

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  3. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  4. 2022Neural Networks and the Chomsky Hierarchycited by 214
  5. 2022Recurrent Memory Transformercited by 197
  6. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  7. 2021Thinking Like Transformerscited by 182
  8. 2023Birth of a Transformer: A Memory Viewpointcited by 154
  9. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
  10. 2021Investigating the Limitations of Transformers with Simple Arithmetic Taskscited by 136
  11. 2023Scaling Transformer to 1M tokens and beyond with RMTcited by 97
  12. 2023Length Generalization in Arithmetic Transformerscited by 64
38 more
  1. 2024Understanding Transformer Reasoning Capabilities via Graph Algorithmscited by 64
  2. 2021Personalized Federated Learning through Local Memorizationcited by 57
  3. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  4. 2021ABC: Attention with Bounded-memory Controlcited by 36
  5. 2022Fine-tuning Image Transformers using Learnable Memorycited by 34
  6. 2024Learning Universal Predictorscited by 29
  7. 2021Artificial Intelligence in Drug Discovery: Applications and Techniquescited by 28
  8. 2021Minimum Description Length Recurrent Neural Networkscited by 23
  9. 2023Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensionscited by 20
  10. 2022Deep Reinforcement Learning for Inverse Inorganic Materials Designcited by 18
  11. 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Modelscited by 18
  12. 2023Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patternscited by 17
  13. 2021Learning Hierarchical Structures with Differentiable Nondeterministic Stackscited by 16
  14. 2022GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesiscited by 16
  15. 2024Memory Mosaicscited by 16
  16. 2024On the Expressiveness and Length Generalization of Selective State-Space Models on Regular Languagescited by 15
  17. 2021UnweaveNet: Unweaving Activity Storiescited by 14
  18. 2023Memory-Based Meta-Learning on Non-Stationary Distributionscited by 14
  19. 2024Bridging the Empirical-Theoretical Gap in Neural Network Formal Language Learning Using Minimum Description Lengthcited by 14
  20. 2022A Call for Clarity in Beam Search: How It Works and When It Stopscited by 10
  21. 2022Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Modelscited by 10
  22. 2024On the Markov Property of Neural Algorithmic Reasoning: Analyses and Methodscited by 10
  23. 2021Can RNNs learn Recursive Nested Subject-Verb Agreements?cited by 9
  24. 2021pix2rule: End-to-end Neuro-symbolic Rule Learningcited by 9
  25. 2023Recursive Algorithmic Reasoningcited by 9
  26. 2021Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Unitscited by 8
  27. 2023On the Computational Complexity and Formal Hierarchy of Second Order Recurrent Neural Networkscited by 8
  28. 2021∞-former: Infinite Memory Transformercited by 7
  29. 2021Learning Bounded Context-Free-Grammar via LSTM and the Transformer:Difference and Explanationscited by 7
  30. 2023Neural Priority Queues for Graph Neural Networkscited by 7
  31. 2024Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Modelscited by 7
  32. 2022The Surprising Computational Power of Nondeterministic Stack RNNscited by 6
  33. 2023Language models in molecular discoverycited by 6
  34. 2019Hierarchical Variational Imitation Learning of Control Programscited by 5
  35. 2021Neural Sequence-to-grid Module for Learning Symbolic Rulescited by 5
  36. 2021Evolutionary Training and Abstraction Yields Algorithmic Generalization of Neural Computerscited by 5
  37. 2023Differentiable Tree Operations Promote Compositional Generalizationcited by 5
  38. 2024Compositional Generalization Across Distributional Shifts with Sparse Tree Operationscited by 5

50 most cited of 76 All 76 (JSON)