Data to 7 Oct 2026

Spectral Filters, Dark Signals, and Attention Sinks

arXiv 2402.09221 ↗First posted 14 Feb 2024cs.AI cs.CLcited by 54

Not on the map. Why?

Abstract (arXiv)

Projecting intermediate representations onto the vocabulary is an increasingly popular interpretation tool for transformer-based LLMs, also known as the logit lens. We propose a quantitative extension to this approach and define spectral filters on intermediate representations based on partitioning the singular vectors of the vocabulary embedding and unembedding matrices into bands. We find that the signals exchanged in the tail end of the spectrum are responsible for attention sinking (Xiao et al. 2023), of which we provide an explanation. We find that the loss of pretrained models can be kept low despite suppressing sizable parts of the embedding spectrum in a layer-dependent way, as long as attention sinking is preserved. Finally, we discover that the representation of tokens that draw attention from many tokens have large projections on the tail end of the spectrum.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 23 of its 35 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

23 of 35 references.

References · 23

2017Attention Is All You Needcited by 59,932
2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
2022Locating and Editing Factual Associations in GPTcited by 2,609
2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
2020GLU Variants Improve Transformercited by 1,724
2022Competition-Level Code Generation with AlphaCodecited by 1,427
2019Root Mean Square Layer Normalizationcited by 1,425
2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
2022In-context Learning and Induction Headscited by 930
2022Toy Models of Superpositioncited by 928

11 more

2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
2023Function Vectors in Large Language Modelscited by 292
2020Information-Theoretic Probing with Minimum Description Lengthcited by 230
2020Information-Theoretic Probing for Linguistic Structurecited by 204
2022Analyzing Transformers in Embedding Spacecited by 131
2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 120
2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 18; 2026: 23.

54 papers.

Sort:

By area:

2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 200
2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 127
2025Why do LLMs attend to the first token?cited by 122
2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 92
2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compressioncited by 84
2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
2024Confidence Regulation Neurons in Language Modelscited by 57
2025Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coincited by 47
2025Mitigating Overthinking in Large Reasoning Models via Manifold Steeringcited by 32

38 more

2025Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Modelscited by 29
2025What are you sinking? A geometric approach on attention sinkcited by 26
2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
2025To Sink or Not to Sink: Visual Information Pathways in Large Vision-Language Modelscited by 18
2025Attention Sinks in Diffusion Language Modelscited by 18
2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17
2025Dense SAE Latents Are Features, Not Bugscited by 17
2026Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Taskscited by 10
2024More Expressive Attention with Negative Weightscited by 9
2025Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMscited by 9
2025Calibration Across Layers: Understanding Calibration Evolution in LLMscited by 9
2025OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inferencecited by 8
2025Don't Deceive Me: Mitigating Gaslighting through Attention Reallocation in LMMscited by 6
2024Frame Representation Hypothesis: Multi-Token LLM Interpretability and Concept-Guided Text Generationcited by 5
2025All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokenscited by 5
2026Why Attention Patterns Exist: A Unifying Temporal Perspective Analysiscited by 4
2024AERO: Entropy-Guided Framework for Private LLM Inferencecited by 3
2025Not a nuisance but a useful heuristic: Outlier dimensions favor frequent tokens in language modelscited by 3
2025On the Existence and Behavior of Secondary Attention Sinkscited by 3
2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 3
2026OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruningcited by 3
2024Investigating the translation capabilities of Large Language Models trained on parallel data onlycited by 2
2026When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Modelscited by 2
2026A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Modelscited by 2
2026Scale Determines Whether Language Models Organize Representation Geometry for Predictioncited by 2
2026Massive Spikes in LLMs are Bias Vectors: Mechanistic Uncovering and Spike-Free Quantizationcited by 2
2024Enhanced Structured State Space Models via Grouped FIR Filtering and Attention Sink Mechanismscited by 1
2026Geometric Analysis of Token Selection in Multi-Head Attentioncited by 1
2026SinkTrack: Attention Sink based Context Anchoring for Large Language Modelscited by 1
2026Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarkscited by 1
2026Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddingscited by 1
2026Why Does Post-Training Quantization Work?cited by 1
2025Sensitivity Meets Sparsity: The Impact of Extremely Sparse Parameter Patterns on Theory-of-Mind of Large Language Modelscited by 0
2026A Mechanistic Account of Attention Sinks in GPT-2: One Circuit, Broader Implications for Mitigationcited by 0
2026The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparitycited by 0
2026A Circuit, Not The Circuit: Non-Unique Causal Localisation of the Mamba-2 State Sinkcited by 0
2026Contribution Weights: A Geometrical Analysis of Self-Attention Transformerscited by 0
2026A Unifying View of Attention Sinks: From Mechanisms to Architectural Interventionscited by 0

50 most cited of 54 All 54 (JSON)