mlsanityData to 7 Oct 2026

Learning to Transduce with Unbounded Memory

arXiv 1506.02516 ↗First posted 8 Jun 2015cs.NE cs.CL cs.LGcited by 54

Not on the map. Why?

Abstract (arXiv)

Recently, strong results have been demonstrated by Deep Recurrent Neural Networks on natural language transduction problems. In this paper we explore the representational power of these models using synthetic grammars designed to exhibit phenomena similar to those found in real transduction problems such as machine translation. These experiments lead us to propose new memory-based recurrent networks that implement continuously differentiable analogues of traditional data structures such as Stacks, Queues, and DeQues. We show that these architectures exhibit superior generalisation performance to Deep RNNs and are often able to learn the underlying generating algorithms in our transduction experiments.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 9; 2023: 15; 2024: 5; 2025: 5; 2026: 3.

54 papers.

Sort:

By area:

  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  3. 2022Neural Networks and the Chomsky Hierarchycited by 214
  4. 2022Recurrent Memory Transformercited by 197
  5. 2023Scaling Transformer to 1M tokens and beyond with RMTcited by 97
  6. 2021Personalized Federated Learning through Local Memorizationcited by 57
  7. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  8. 2021ABC: Attention with Bounded-memory Controlcited by 36
  9. 2021Lexicon Learning for Few-Shot Neural Sequence Modelingcited by 28
  10. 2023Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensionscited by 20
  11. 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Modelscited by 18
  12. 2023Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patternscited by 17
38 more
  1. 2021Learning Hierarchical Structures with Differentiable Nondeterministic Stackscited by 16
  2. 2021Modeling Hierarchical Structures with Continuous Recursive Neural Networkscited by 15
  3. 2021UnweaveNet: Unweaving Activity Storiescited by 14
  4. 2024Autoregressive Large Language Models are Computationally Universalcited by 14
  5. 2022A Call for Clarity in Beam Search: How It Works and When It Stopscited by 10
  6. 2021Formal Language Theory Meets Modern NLPcited by 9
  7. 2021Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Unitscited by 8
  8. 2021Learning to Navigate Intersections with Unsupervised Driver Trait Inferencecited by 8
  9. 2022Characterizing Verbatim Short-Term Memory in Neural Language Modelscited by 8
  10. 2023On the Computational Complexity and Formal Hierarchy of Second Order Recurrent Neural Networkscited by 8
  11. 2021∞-former: Infinite Memory Transformercited by 7
  12. 2023Neural Priority Queues for Graph Neural Networkscited by 7
  13. 2022The Surprising Computational Power of Nondeterministic Stack RNNscited by 6
  14. 2023Beam Tree Recursive Cellscited by 6
  15. 2021Neural Sequence-to-grid Module for Learning Symbolic Rulescited by 5
  16. 2021Evolutionary Training and Abstraction Yields Algorithmic Generalization of Neural Computerscited by 5
  17. 2023Differentiable Tree Operations Promote Compositional Generalizationcited by 5
  18. 2024Compositional Generalization Across Distributional Shifts with Sparse Tree Operationscited by 5
  19. 2023Theoretical Conditions and Empirical Failure of Bracket Counting on Long Sequences with Linear Recurrent Networkscited by 3
  20. 2022State-Regularized Recurrent Neural Networks to Extract Automata and Explain Predictionscited by 2
  21. 2023A Framework for Inference Inspired by Human Memory Mechanismscited by 2
  22. 2024A Transformer with Stack Attentioncited by 2
  23. 2021Reconciling the Discrete-Continuous Divide: Towards a Mathematical Theory of Sparse Communicationcited by 1
  24. 2021Learning Complex Users' Preferences for Recommender Systemscited by 1
  25. 2023Neural Attention Memorycited by 1
  26. 2023Dynamic Analysis and an Eigen Initializer for Recurrent Neural Networkscited by 1
  27. 2023Universal Recurrent Event Memories for Streaming Datacited by 1
  28. 2025An Expert Ensemble for Detecting Anomalous Scenes, Interactions, and Behaviors in Autonomous Drivingcited by 1
  29. 2025Exact Learning of Arithmetic with Differentiable Agentscited by 1
  30. 2026Universal computation is intrinsic to language model decodingcited by 1
  31. 2021A Neural Conversation Generation Model via Equivalent Shared Memory Investigationcited by 0
  32. 2021Enhancing Reinforcement Learning with discrete interfaces to learn the Dyck Languagecited by 0
  33. 2022Differentiable Quantum Programming with Unbounded Loopscited by 0
  34. 2023Learning Transductions and Alignments with RNN Seq2seq Modelscited by 0
  35. 2023Style Locality for Controllable Generation with kNN Language Modelscited by 0
  36. 2024Enhancing Length Extrapolation in Sequential Models with Pointer-Augmented Neural Memorycited by 0
  37. 2024Exploring Learnability in Memory-Augmented Recurrent Neural Networks: Precision, Stability, and Empirical Insightscited by 0
  38. 2025Why Are Parsing Actions for Understanding Message Hierarchies Not Random?cited by 0

50 most cited of 54 All 54 (JSON)