# Correlating neural and symbolic representations of language

arXiv: https://arxiv.org/abs/1905.06401 · HTML: https://mlsanity.com/paper/1905.06401/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2019.

## Built on by (most cited first, 30 of 33)

- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality](https://mlsanity.com/paper/2109.04404/) (2021)
- [On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation](https://mlsanity.com/paper/2106.03164/) (2021)
- [A Closer Look at How Fine-tuning Changes BERT](https://mlsanity.com/paper/2106.14282/) (2021)
- [MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment](https://mlsanity.com/paper/2410.05873/) (2024)
- [Unit Testing for Concepts in Neural Networks](https://mlsanity.com/paper/2208.10244/) (2022)
- [Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?](https://mlsanity.com/paper/2205.10226/) (2022)
- [Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond](https://mlsanity.com/paper/2304.05216/) (2023)
- [A comparative evaluation and analysis of three generations of Distributional Semantic Models](https://mlsanity.com/paper/2105.09825/) (2021)
- [Wave to Syntax: Probing spoken language models for syntax](https://mlsanity.com/paper/2305.18957/) (2023)
- [Local Structure Matters Most: Perturbation Study in NLU](https://mlsanity.com/paper/2107.13955/) (2021)
- [Neuro-Symbolic AI: Explainability, Challenges, and Future Trends](https://mlsanity.com/paper/2411.04383/) (2024)
- [Do Large Language Models Mirror Cognitive Language Processing?](https://mlsanity.com/paper/2402.18023/) (2024)
- [How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddings](https://mlsanity.com/paper/2109.10179/) (2021)
- [Decoding In-Context Learning: Neuroscience-inspired Analysis of Representations in Large Language Models](https://mlsanity.com/paper/2310.00313/) (2023)
- [Linguistic Interpretability of Transformer-based Language Models: a systematic review](https://mlsanity.com/paper/2504.08001/) (2025)
- [Large Language Models are not Models of Natural Language: they are Corpus Models](https://mlsanity.com/paper/2112.07055/) (2021)
- [One Size Does Not Fit All: Finding the Optimal Subword Sizes for FastText Models across Languages](https://mlsanity.com/paper/2102.02585/) (2021)
- [Discrete representations in neural models of spoken language](https://mlsanity.com/paper/2105.05582/) (2021)
- [How Does Adversarial Fine-Tuning Benefit BERT?](https://mlsanity.com/paper/2108.13602/) (2021)
- [Quantifying Compositionality of Classic and State-of-the-Art Embeddings](https://mlsanity.com/paper/2509.19332/) (2025)
- [Abstraction not Memory: BERT and the English Article System](https://mlsanity.com/paper/2206.04184/) (2022)
- [Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings](https://mlsanity.com/paper/2301.08998/) (2023)
- [Semantic Composition in Visually Grounded Language Models](https://mlsanity.com/paper/2305.16328/) (2023)
- [Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models](https://mlsanity.com/paper/2308.08774/) (2023)
- [How Language Models Prioritize Contextual Grammatical Cues?](https://mlsanity.com/paper/2410.03447/) (2024)
- [The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models](https://mlsanity.com/paper/2601.19926/) (2026)
- [We went to look for meaning and all we got were these lousy representations: aspects of meaning representation for computational semantics](https://mlsanity.com/paper/2109.04949/) (2021)
- [Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making](https://mlsanity.com/paper/2305.17588/) (2023)

## Abstract (arXiv)

Analysis methods which enable us to better understand the representations and functioning of neural models of language are increasingly needed as deep learning becomes the dominant approach in NLP. Here we present two methods based on Representational Similarity Analysis (RSA) and Tree Kernels (TK) which allow us to directly quantify how strongly the information encoded in neural activation patterns corresponds to information represented by symbolic structures such as syntax trees. We first validate our methods on the case of a simple synthetic language for arithmetic expressions with clearly defined syntax and semantics, and show that they exhibit the expected pattern of results. We then apply our methods to correlate neural representations of English sentences with their constituency parse trees.