# Decomposing Query-Key Feature Interactions Using Contrastive Covariances

arXiv: https://arxiv.org/abs/2602.04752 · HTML: https://mlsanity.com/paper/2602.04752/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Builds on (most cited first, 30 of 31)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [Not All Language Model Features Are One-Dimensionally Linear](https://mlsanity.com/paper/2405.14860/) (2024)
- [Round and Round We Go! What makes Rotary Positional Encodings useful?](https://mlsanity.com/paper/2410.06205/) (2024)
- [How do Language Models Bind Entities in Context?](https://mlsanity.com/paper/2310.17191/) (2023)
- [Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models](https://mlsanity.com/paper/1804.09299/) (2018)
- [VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers](https://mlsanity.com/paper/2203.17247/) (2022)
- [AttentionViz: A Global View of Transformer Attention](https://mlsanity.com/paper/2305.03210/) (2023)
- [When Models Manipulate Manifolds: The Geometry of a Counting Task](https://mlsanity.com/paper/2601.04480/) (2026)
- [Talking Heads: Understanding Inter-layer Communication in Transformer Language Models](https://mlsanity.com/paper/2406.09519/) (2024)
- [Language Models use Lookbacks to Track Beliefs](https://mlsanity.com/paper/2505.14685/) (2025)
- [Representational Analysis of Binding in Language Models](https://mlsanity.com/paper/2409.05448/) (2024)
- [Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Context](https://mlsanity.com/paper/2510.06182/) (2025)
- [Automatically Identifying Local and Global Circuits with Linear Computation Graphs](https://mlsanity.com/paper/2405.13868/) (2024)
- [Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers?](https://mlsanity.com/paper/2510.24709/) (2025)
- [The Geometry of Self-Verification in a Task-Specific Reasoning Model](https://mlsanity.com/paper/2504.14379/) (2025)
- [Dissecting Query-Key Interaction in Vision Transformers](https://mlsanity.com/paper/2405.14880/) (2024)
- [Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning](https://mlsanity.com/paper/2508.01916/) (2025)
- [LLMs Process Lists With General Filter Heads](https://mlsanity.com/paper/2510.26784/) (2025)
- [Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits](https://mlsanity.com/paper/2511.20273/) (2025)
- [Bi-Orthogonal Factor Decomposition for Vision Transformers](https://mlsanity.com/paper/2601.05328/) (2026)

## Built on by (most cited first, 3 of 3)

- [Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions](https://mlsanity.com/paper/2605.09967/) (2026)
- [MAxBench: A Multinomial Concept Recovery Benchmark](https://mlsanity.com/paper/2609.13072/) (2026)
- [Using OCR Heads to Verbalize Image Semantics](https://mlsanity.com/paper/2609.18823/) (2026)

## Abstract (arXiv)

Despite the central role of attention heads in Transformers, we lack tools to understand why a model attends to a particular token. To address this, we study the query-key (QK) space -- the bilinear joint embedding space between queries and keys. We present a contrastive covariance method to decompose the QK space into low-rank, human-interpretable components. It is when features in keys and queries align in these low-rank subspaces that high attention scores are produced. We first study our method both analytically and empirically in a simplified setting. We then apply our method to large language models to identify human-interpretable QK subspaces for categorical semantic features and binding features. Finally, we demonstrate how attention scores can be attributed to our identified features.