mlsanityData to 14 Oct 2026

Outlier Dimensions Encode Task-Specific Knowledge

arXiv 2310.17715 ↗First posted 26 Oct 2023cs.CL cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

Representations from large language models (LLMs) are known to be dominated by a small subset of dimensions with exceedingly high variance. Previous works have argued that although ablating these outlier dimensions in LLM representations hurts downstream performance, outlier dimensions are detrimental to the representational quality of embeddings. In this study, we investigate how fine-tuning impacts outlier dimensions and show that 1) outlier dimensions that occur in pre-training persist in fine-tuned models and 2) a single outlier dimension can complete downstream tasks with a minimal error rate. Our results suggest that outlier dimensions can encode crucial task-specific knowledge and that the value of a representation in a single outlier dimension drives downstream model decisions.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

24 of 36 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  7. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  8. 2018Neural Network Acceptability Judgmentscited by 958
  9. 2019Intrinsic dimension of data representations in deep neural networkscited by 282
  10. 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 255
  11. 2019Visualizing and Measuring the Geometry of BERTcited by 216
  12. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 212
12 more
  1. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 185
  2. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 156
  3. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  4. 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 112
  5. 2022Cramming: Training a Language Model on a Single GPU in One Daycited by 81
  6. 2022Outliers Dimensions that Disrupt Transformers Are Driven by Frequencycited by 55
  7. 2021IsoScore: Measuring the Uniformity of Embedding Space Utilizationcited by 53
  8. 2021The Low-Dimensional Linear Geometry of Contextualized Word Representationscited by 40
  9. 2019Dimensionality compression and expansion in Deep Neural Networkscited by 38
  10. 2021A Cluster-based Approach for Improving Isotropy in Contextual Embedding Spacecited by 27
  11. 2021Learning to Remove: Towards Isotropic Pre-trained BERT Embeddingcited by 23
  12. 2020IsoBN: Fine-Tuning BERT with Isotropic Batch Normalizationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 5.

15 papers.

Sort:

By area:

  1. 2024What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluationcited by 30
  2. 2024Unveiling Linguistic Regions in Large Language Modelscited by 22
  3. 2024Geometric Signatures of Compositionality Across a Language Model's Lifetimecited by 16
  4. 2025Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddingscited by 16
  5. 2025Randomly Removing 50% of Dimensions in Text Embeddings has Minimal Impact on Retrieval and Classification Taskscited by 8
  6. 2025Model Internal Sleuthing: Finding Lexical Identity and Inflectional Features in Modern Language Modelscited by 5
  7. 2025Mining Useful General Data for Low-Resource Domain Adaptationcited by 4
  8. 2025SEAL: Scaling to Emphasize Attention for Long-Context Retrievalcited by 2
  9. 2026Hista and Numca: Estimate State Value Effectively for LLM Reinforcement Learningcited by 2
  10. 2025TRIM: Achieving Extreme Sparsity with Targeted Row-wise Iterative Metric-driven Pruningcited by 1
  11. 2025ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attackcited by 1
  12. 2026Towards Large Model Feature Codingcited by 1
3 more
  1. 2026Embracing Anisotropy: Turning Massive Activations into Interpretable Control Knobs for Large Language Modelscited by 0
  2. 2026Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Modelscited by 0
  3. 2026Decoupling Internal Representational Changes and Causal Importance in Fine-Tuned Large Language Modelscited by 0