# Mechanistic Decomposition of Sentence Representations

arXiv: https://arxiv.org/abs/2506.04373 · HTML: https://mlsanity.com/paper/2506.04373/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 30 of 32)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://mlsanity.com/paper/2407.19669/) (2024)
- [Information-Theoretic Probing with Minimum Description Length](https://mlsanity.com/paper/2003.12298/) (2020)
- [Information-Theoretic Probing for Linguistic Structure](https://mlsanity.com/paper/2004.03061/) (2020)
- [Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?](https://mlsanity.com/paper/2005.00719/) (2020)
- [Probing Across Time: What Does RoBERTa Know and When?](https://mlsanity.com/paper/2104.07885/) (2021)
- [Towards Universal Paraphrastic Sentence Embeddings](https://mlsanity.com/paper/1511.08198/) (2015)
- [Information Retrieval: Recent Advances and Beyond](https://mlsanity.com/paper/2301.08801/) (2023)
- [A Survey on Contextual Embeddings](https://mlsanity.com/paper/2003.07278/) (2020)
- [SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features](https://mlsanity.com/paper/2206.07023/) (2022)
- [Core Semantic First: A Top-down Approach for AMR Parsing](https://mlsanity.com/paper/1909.04303/) (2019)
- [Comparative Analysis of Pooling Mechanisms in LLMs: A Sentiment Analysis Perspective](https://mlsanity.com/paper/2411.14654/) (2024)
- [Interpretable Text Embeddings and Text Similarity Explanation: A Survey](https://mlsanity.com/paper/2502.14862/) (2025)
- [Explaining Text Similarity in Transformer Models](https://mlsanity.com/paper/2405.06604/) (2024)
- [An Attribution Method for Siamese Encoders](https://mlsanity.com/paper/2310.05703/) (2023)
- [Approximate Attributions for Off-the-Shelf Siamese Transformers](https://mlsanity.com/paper/2402.02883/) (2024)
- [Tracking linguistic information in transformer-based sentence embeddings through targeted sparsification](https://mlsanity.com/paper/2407.18119/) (2024)

## Abstract (arXiv)

Sentence embeddings are central to modern NLP and AI systems, yet little is known about their internal structure. While we can compare these embeddings using measures such as cosine similarity, the contributing features are not human-interpretable, and the content of an embedding seems untraceable, as it is masked by complex neural transformations and a final pooling operation that combines individual token embeddings. To alleviate this issue, we propose a new method to mechanistically decompose sentence embeddings into interpretable components, by using dictionary learning on token-level representations. We analyze how pooling compresses these features into sentence representations, and assess the latent features that reside in a sentence embedding. This bridges token-level mechanistic interpretability with sentence-level analysis, making for more transparent and controllable representations. In our studies, we obtain several interesting insights into the inner workings of sentence embedding spaces, for instance, that many semantic and syntactic aspects are linearly encoded in the embeddings.