mlsanityData to 7 Oct 2026

Correlating neural and symbolic representations of language

arXiv 1905.06401 ↗First posted 14 May 2019cs.CL cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Analysis methods which enable us to better understand the representations and functioning of neural models of language are increasingly needed as deep learning becomes the dominant approach in NLP. Here we present two methods based on Representational Similarity Analysis (RSA) and Tree Kernels (TK) which allow us to directly quantify how strongly the information encoded in neural activation patterns corresponds to information represented by symbolic structures such as syntax trees. We first validate our methods on the case of a simple synthetic language for arithmetic expressions with clearly defined syntax and semantics, and show that they exhibit the expected pattern of results. We then apply our methods to correlate neural representations of English sentences with their constituency parse trees.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 3; 2023: 7; 2024: 6; 2025: 2; 2026: 2.

32 papers.

Sort:

By area:

  1. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  2. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 186
  3. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
  4. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  5. 2021A Closer Look at How Fine-tuning Changes BERTcited by 42
  6. 2024MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentcited by 25
  7. 2022Unit Testing for Concepts in Neural Networkscited by 23
  8. 2022Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?cited by 20
  9. 2023Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyondcited by 20
  10. 2021A comparative evaluation and analysis of three generations of Distributional Semantic Modelscited by 19
  11. 2023Wave to Syntax: Probing spoken language models for syntaxcited by 18
  12. 2021Local Structure Matters Most: Perturbation Study in NLUcited by 10
20 more
  1. 2024Neuro-Symbolic AI: Explainability, Challenges, and Future Trendscited by 10
  2. 2024Do Large Language Models Mirror Cognitive Language Processing?cited by 9
  3. 2021How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddingscited by 8
  4. 2023Decoding In-Context Learning: Neuroscience-inspired Analysis of Representations in Large Language Modelscited by 7
  5. 2025Linguistic Interpretability of Transformer-based Language Models: a systematic reviewcited by 7
  6. 2021Large Language Models are not Models of Natural Language: they are Corpus Modelscited by 6
  7. 2021One Size Does Not Fit All: Finding the Optimal Subword Sizes for FastText Models across Languagescited by 4
  8. 2021Discrete representations in neural models of spoken languagecited by 3
  9. 2021How Does Adversarial Fine-Tuning Benefit BERT?cited by 3
  10. 2025Quantifying Compositionality of Classic and State-of-the-Art Embeddingscited by 2
  11. 2022Abstraction not Memory: BERT and the English Article Systemcited by 1
  12. 2023Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddingscited by 1
  13. 2023Semantic Composition in Visually Grounded Language Modelscited by 1
  14. 2023Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Modelscited by 1
  15. 2024How Language Models Prioritize Contextual Grammatical Cues?cited by 1
  16. 2026The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Modelscited by 1
  17. 2021We went to look for meaning and all we got were these lousy representations: aspects of meaning representation for computational semanticscited by 0
  18. 2023Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Makingcited by 0
  19. 2024Bridging the Gap: Representation Spaces in Neuro-Symbolic AIcited by 0
  20. 2026Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probecited by 0