# A framework for auditing grounding claims

arXiv: https://arxiv.org/abs/2512.06205 · HTML: https://mlsanity.com/paper/2512.06205/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 87 references that are in the map.
- First posted on arXiv 5 Dec 2025.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [COGS: A Compositional Generalization Challenge Based on Semantic Interpretation](https://mlsanity.com/paper/2010.05465/) (2020)
- [RVT: Robotic View Transformer for 3D Object Manipulation](https://mlsanity.com/paper/2306.14896/) (2023)
- [Implicit Representations of Meaning in Neural Language Models](https://mlsanity.com/paper/2106.00737/) (2021)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [Evaluating the Robustness of Neural Language Models to Input Perturbations](https://mlsanity.com/paper/2108.12237/) (2021)
- [The Vector Grounding Problem](https://mlsanity.com/paper/2304.01481/) (2023)
- [Understanding the Emergence of Multimodal Representation Alignment](https://mlsanity.com/paper/2502.16282/) (2025)
- [How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy](https://mlsanity.com/paper/2109.04740/) (2021)
- [Resilience of Large Language Models for Noisy Instructions](https://mlsanity.com/paper/2404.09754/) (2024)
- [Do Language Models' Words Refer?](https://mlsanity.com/paper/2308.05576/) (2023)
- [Robustness of Large Language Models to Perturbations in Text](https://mlsanity.com/paper/2407.08989/) (2024)
- [Entailment-Preserving First-order Logic Representations in Natural Language Entailment](https://mlsanity.com/paper/2502.16757/) (2025)
- [On the Robustness of Text Vectorizers](https://mlsanity.com/paper/2303.07203/) (2023)

## Abstract (arXiv)

The symbol grounding problem asks how a token such as cat can be about cats. We propose a framework for auditing grounding claims against a declared semantic standard. The audit reports measurements and evidence, with overall verdicts conditional on explicit acceptance criteria. Its profiles assess accuracy, robustness, and composition alongside evidence about how the system acquired its mechanisms, how they contribute to performance, and why they were retained. In a toy gridworld, an agent interprets individual symbols accurately but fails a withheld combination. Composing its interpretations by the declared rule would succeed. This comparison identifies a departure from the composition rule within the observed failure. Both this audit and a pilot on pretrained word vectors provide evidence that a designated mechanism contributes to present performance. Whether that contribution explains its retention remains uncertified. The framework evaluates the evidence for grounding claims; candidate accounts remain responsible for explaining how meaning emerges.