mlsanityData to 7 Oct 2026

Discovering the Compositional Structure of Vector Representations with Role Learning Networks

arXiv 1910.09113 ↗First posted 21 Oct 2019cs.LG cs.CL stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

How can neural networks perform so well on compositional tasks even though they lack explicit compositional representations? We use a novel analysis technique called ROLE to show that recurrent neural networks perform well on such tasks by converging to solutions which implicitly represent symbolic structure. This method uncovers a symbolic structure which, when properly embedded in vector space, closely approximates the encodings of a standard seq2seq network trained to perform the compositional SCAN task. We verify the causal importance of the discovered symbolic structure by showing that, when we systematically manipulate hidden embeddings based on this symbolic structure, the model's output is changed in the way predicted by our analysis.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 8; 2024: 6; 2025: 4; 2026: 3.

33 papers.

Sort:

  1. 2021Causal Abstractions of Neural Networkscited by 401
  2. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  3. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  4. 2024How to use and interpret activation patchingcited by 191
  5. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  6. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  7. 2022CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behaviorcited by 54
  8. 2022Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systemscited by 44
  9. 2022Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionalitycited by 44
  10. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  11. 2022Causal Proxy Models for Concept-Based Model Explanationscited by 29
  12. 2021Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probingcited by 28
21 more
  1. 2024A polar coordinate system represents syntax in large language modelscited by 24
  2. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 21
  3. 2021Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarizationcited by 11
  4. 2025RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patchingcited by 10
  5. 2022Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Modelscited by 9
  6. 2023Causal interventions expose implicit situation models for commonsense language understandingcited by 7
  7. 2024From Frege to chatGPT: Compositionality in language, cognition, and deep neural networkscited by 7
  8. 2022Structural Biases for Improving Transformers on Translation into Morphologically Rich Languagescited by 5
  9. 2023Differentiable Tree Operations Promote Compositional Generalizationcited by 5
  10. 2024Compositional Generalization Across Distributional Shifts with Sparse Tree Operationscited by 5
  11. 2025Distinct Computations Emerge From Compositional Curricula in In-Context Learningcited by 5
  12. 2025Identifying and Mitigating the Influence of the Prior Distribution in Large Language Modelscited by 4
  13. 2026From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Taskscited by 3
  14. 2021Distributionally Robust Recurrent Decoders with Random Network Distillationcited by 2
  15. 2021General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddingscited by 1
  16. 2023Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddingscited by 1
  17. 2023Pretrained Embeddings for E-commerce Machine Learning: When it Fails and Why?cited by 1
  18. 2023Semantic Composition in Visually Grounded Language Modelscited by 1
  19. 2026Steer Like the LLM: Activation Steering that Mimics Promptingcited by 1
  20. 2025Kronecker Factorization Improves Efficiency and Interpretability of Sparse Autoencoderscited by 0
  21. 2026A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretabilitycited by 0