mlsanityData to 7 Oct 2026

Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages

arXiv 1911.03329 ↗First posted 8 Nov 2019cs.CL cs.LG cs.NEcited by 31

Not on the map. Why?

Abstract (arXiv)

We introduce three memory-augmented Recurrent Neural Networks (MARNNs) and explore their capabilities on a series of simple language modeling tasks whose solutions require stack-based mechanisms. We provide the first demonstration of neural networks recognizing the generalized Dyck languages, which express the core of what it means to be a language with hierarchical structure. Our memory-augmented architectures are easy to train in an end-to-end fashion and can learn the Dyck languages over as many as six parenthesis-pairs, in addition to two deterministic palindrome languages and the string-reversal transduction task, by emulating pushdown automata. Our experiments highlight the increased modeling capacity of memory-augmented models over simple RNNs, while inflecting our understanding of the limitations of these models.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 1; 2023: 6; 2024: 8; 2025: 5; 2026: 1.

31 papers.

Sort:

By area:

  1. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  2. 2022Neural Networks and the Chomsky Hierarchycited by 214
  3. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
  4. 2021Self-Attention Networks Can Process Bounded Hierarchical Languagescited by 105
  5. 2024In-Context Language Learning: Architectures and Algorithmscited by 86
  6. 2024Learning Universal Predictorscited by 29
  7. 2024Separations in the Representational Capabilities of Transformers and Recurrent Architecturescited by 29
  8. 2021Minimum Description Length Recurrent Neural Networkscited by 23
  9. 2024Training Neural Networks as Recognizers of Formal Languagescited by 23
  10. 2023Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensionscited by 20
  11. 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Modelscited by 18
  12. 2021Learning Hierarchical Structures with Differentiable Nondeterministic Stackscited by 16
19 more
  1. 2024On the Expressiveness and Length Generalization of Selective State-Space Models on Regular Languagescited by 15
  2. 2024Emergent Word Order Universals from Cognitively-Motivated Language Modelscited by 11
  3. 2021Can RNNs learn Recursive Nested Subject-Verb Agreements?cited by 9
  4. 2021Formal Language Theory Meets Modern NLPcited by 9
  5. 2021Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Unitscited by 8
  6. 2021Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humanscited by 8
  7. 2023Feature Interactions Reveal Linguistic Structure in Language Modelscited by 8
  8. 2025Can Interpretation Predict Behavior on Unseen Data?cited by 8
  9. 2021Learning Bounded Context-Free-Grammar via LSTM and the Transformer:Difference and Explanationscited by 7
  10. 2023Theoretical Conditions and Empirical Failure of Bracket Counting on Long Sequences with Linear Recurrent Networkscited by 3
  11. 2023Benchmarking Neural Network Generalization for Grammar Inductioncited by 3
  12. 2024A Transformer with Stack Attentioncited by 2
  13. 2025Emergent Stack Representations in Modeling Counter Languages Using Transformerscited by 2
  14. 2021State-Space Constraints Can Improve the Generalisation of the Differentiable Neural Computer to Input Sequences With Unseen Lengthcited by 1
  15. 2026On the Emergence of Syntax by Means of Local Interactioncited by 1
  16. 2021Enhancing Reinforcement Learning with discrete interfaces to learn the Dyck Languagecited by 0
  17. 2023Nondeterministic Stacks in Neural Networkscited by 0
  18. 2024Enhancing Length Extrapolation in Sequential Models with Pointer-Augmented Neural Memorycited by 0
  19. 2025MapFormer: Self-Supervised Learning of Cognitive Maps with Input-Dependent Positional Embeddingscited by 0