mlsanityData to 7 Oct 2026

The Importance of Being Recurrent for Modeling Hierarchical Structure

arXiv 1803.03585 ↗First posted 9 Mar 2018cs.CLcited by 42

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that recurrent neural networks (RNNs) can implicitly capture and exploit hierarchical information when trained to solve common natural language processing tasks such as language modeling (Linzen et al., 2016) and neural machine translation (Shi et al., 2016). In contrast, the ability to model structured data with non-recurrent neural networks has received little attention despite their success in many NLP tasks (Gehring et al., 2017; Vaswani et al., 2017). In this work, we compare the two architectures---recurrent versus non-recurrent---with respect to their ability to model hierarchical structure and find that recurrency is indeed important for this purpose.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 6; 2023: 10; 2024: 5; 2025: 2; 2026: 4.

42 papers.

Sort:

By area:

  1. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  2. 2021The Benchmark Lotterycited by 133
  3. 2021Self-Attention Networks Can Process Bounded Hierarchical Languagescited by 105
  4. 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
  5. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  6. 2023Grokking of Hierarchical Structure in Vanilla Transformerscited by 53
  7. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  8. 2021Refining Targeted Syntactic Evaluation of Language Modelscited by 41
  9. 2022On the Explainability of Natural Language Processing Deep Modelscited by 37
  10. 2022Pretraining with Artificial Language: Studying Transferable Knowledge in Language Modelscited by 28
  11. 2023Sparse Universal Transformercited by 28
  12. 2021Modeling Hierarchical Structures with Continuous Recursive Neural Networkscited by 15
30 more
  1. 2023Can LLMs facilitate interpretation of pre-trained language models?cited by 15
  2. 2021Evaluating Document Coherence Modellingcited by 8
  3. 2021On the Difficulty of Translating Free-Order Case-Marking Languagescited by 8
  4. 2025Can Interpretation Predict Behavior on Unseen Data?cited by 8
  5. 2023Beam Tree Recursive Cellscited by 6
  6. 2023TAPIR: Learning Adaptive Revision for Incremental Natural Language Understanding with a Two-Pass Modelcited by 5
  7. 2023Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalabilitycited by 5
  8. 2024Investigating Recurrent Transformers with Dynamic Haltcited by 5
  9. 2021Fine-grained Interpretation and Causation Analysis in Deep NLP Modelscited by 4
  10. 2022Implicit N-grams Induced by Recurrencecited by 4
  11. 2023Efficient Beam Tree Recursioncited by 4
  12. 2024AI and Generative AI for Research Discovery and Summarizationcited by 4
  13. 2022A computational psycholinguistic evaluation of the syntactic abilities of Galician BERT models at the interface of dependency resolution and training timecited by 3
  14. 2022Forming Trees with Treeformerscited by 3
  15. 2021Can Edge Probing Tasks Reveal Linguistic Knowledge in QA Models?cited by 2
  16. 2023Tracing and Manipulating Intermediate Values in Neural Math Problem Solverscited by 2
  17. 2021Vy=akarana: A Colorless Green Benchmark for Syntactic Evaluation in Indic Languagescited by 1
  18. 2021ORCHARD: A Benchmark For Measuring Systematic Generalization of Multi-Hierarchical Reasoningcited by 1
  19. 2023Implications of Annotation Artifacts in Edge Probing Test Datasetscited by 1
  20. 2024On the Design Space Between Transformers and Recursive Neural Netscited by 1
  21. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1
  22. 2021To Understand Representation of Layer-aware Sequence Encoders as Multi-order-graphcited by 0
  23. 2021An empirical analysis of phrase-based and neural machine translationcited by 0
  24. 2022Syntactic Inductive Biases for Deep Learning Methodscited by 0
  25. 2023Comparing Generalization in Learning with Limited Numbers of Exemplars: Transformer vs. RNN in Attractor Dynamicscited by 0
  26. 2024Hierarchical Structure Enhances the Convergence and Generalizability of Linear Molecular Representationcited by 0
  27. 2025Who invented deep residual learning?cited by 0
  28. 2026Dependency Parsing Across the Resource Spectrum: Evaluating Architectures on High and Low-Resource Languagescited by 0
  29. 2026Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networkscited by 0
  30. 2026Language Re-generation: An investigation into information locality effects on reconstructioncited by 0