# Why bother with geometry? On the relevance of linear decompositions of Transformer embeddings

arXiv: https://arxiv.org/abs/2310.06977 · HTML: https://mlsanity.com/paper/2310.06977/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Marian: Fast Neural Machine Translation in C++](https://mlsanity.com/paper/1804.00344/) (2018)
- [All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality](https://mlsanity.com/paper/2109.04404/) (2021)
- [Analyzing Transformers in Embedding Space](https://mlsanity.com/paper/2209.02535/) (2022)
- [The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT](https://mlsanity.com/paper/2010.06354/) (2020)
- [Learning to Deceive with Attention-Based Explanations](https://mlsanity.com/paper/1909.07913/) (2019)
- [Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation](https://mlsanity.com/paper/2010.10907/) (2020)
- [Measuring the Mixing of Contextual Information in the Transformer](https://mlsanity.com/paper/2203.04212/) (2022)
- [Analogies Explained: Towards Understanding Word Embeddings](https://mlsanity.com/paper/1901.09813/) (2019)
- [Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer](https://mlsanity.com/paper/2205.11631/) (2022)
- [GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers](https://mlsanity.com/paper/2205.03286/) (2022)
- [Explaining How Transformers Use Context to Build Predictions](https://mlsanity.com/paper/2305.12535/) (2023)
- [How to Dissect a Muppet: The Structure of Transformer Embedding Spaces](https://mlsanity.com/paper/2206.03529/) (2022)
- [Of Non-Linearity and Commutativity in BERT](https://mlsanity.com/paper/2101.04547/) (2021)
- [Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictions](https://mlsanity.com/paper/2305.10614/) (2023)

## Built on by (most cited first, 2 of 2)

- [SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes](https://mlsanity.com/paper/2403.07726/) (2024)
- [Isotropy, Clusters, and Classifiers](https://mlsanity.com/paper/2402.03191/) (2024)

## Abstract (arXiv)

A recent body of work has demonstrated that Transformer embeddings can be linearly decomposed into well-defined sums of factors, that can in turn be related to specific network inputs or components. There is however still a dearth of work studying whether these mathematical reformulations are empirically meaningful. In the present work, we study representations from machine-translation decoders using two of such embedding decomposition methods. Our results indicate that, while decomposition-derived indicators effectively correlate with model performance, variation across different runs suggests a more nuanced take on this question. The high variability of our measurements indicate that geometry reflects model-specific characteristics more than it does sentence-specific computations, and that similar training conditions do not guarantee similar vector spaces.