mlsanityData to 14 Oct 2026

Outliers Dimensions that Disrupt Transformers Are Driven by Frequency

arXiv 2205.11380 ↗First posted 23 May 2022cs.CL cs.AIcited by 55

Not on the map. Why?

Abstract (arXiv)

While Transformer-based language models are generally very robust to pruning, there is the recently discovered outlier phenomenon: disabling only 48 out of 110M parameters in BERT-base drops its performance by nearly 30% on MNLI. We replicate the original evidence for the outlier phenomenon and we link it to the geometry of the embedding space. We find that in both BERT and RoBERTa the magnitude of hidden state coefficients corresponding to outlier dimensions correlates with the frequency of encoded tokens in pre-training data, and it also contributes to the "vertical" self-attention pattern enabling the model to focus on the special tokens. This explains the drop in performance from disabling the outliers, and it suggests that to decrease anisotropicity in future models we need pre-training schemas that would better take into account the skewed token distributions.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

31 of 44 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  7. 2016Pointer Sentinel Mixture Modelscited by 3,096
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  9. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  10. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  11. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  12. 2018Neural Network Acceptability Judgmentscited by 958
19 more
  1. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  2. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 419
  3. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 403
  4. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
  5. 2019Revealing the Dark Secrets of BERTcited by 301
  6. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  7. 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 255
  8. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 185
  9. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 156
  10. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 141
  11. 2019Well-Read Students Learn Better: On the Importance of Pre-training Compact Modelscited by 120
  12. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  13. 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 112
  14. 2021The MultiBERTs: BERT Reproductions for Robustness Analysiscited by 92
  15. 2020Positional Artefacts Propagate Through Masked Language Model Embeddingscited by 35
  16. 2021How is BERT surprised? Layerwise detection of linguistic anomaliescited by 24
  17. 2021Learning to Remove: Towards Isotropic Pre-trained BERT Embeddingcited by 23
  18. 2021How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropycited by 23
  19. 2021Frequency-aware SGD for Efficient Embedding Learning with Provable Benefitscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 11; 2024: 18; 2025: 11; 2026: 11.

55 papers.

Sort:

By area:

  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  2. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  3. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  4. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  5. 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 165
  6. 2024Anisotropy Is Inherent to Self-Attention in Transformerscited by 63
  7. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  8. 2023Intriguing Properties of Quantization at Scalecited by 49
  9. 2024Scalable Influence and Fact Tracing for Large Language Model Pretrainingcited by 42
  10. 2024AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Modelscited by 36
  11. 2024Understanding Cross-Lingual Alignment -- A Surveycited by 34
  12. 2023Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Mapscited by 33
38 more
  1. 2023VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformerscited by 32
  2. 2024Hyperbolic Fine-Tuning for Large Language Modelscited by 27
  3. 2026The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinkscited by 27
  4. 2024Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularizationcited by 23
  5. 2026A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Trainingcited by 23
  6. 2024Unveiling Linguistic Regions in Large Language Modelscited by 22
  7. 2024Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneckcited by 21
  8. 2024Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMscited by 21
  9. 2023Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Modelscited by 19
  10. 2024Robust AI-Generated Text Detection by Restricted Embeddingscited by 18
  11. 2023Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similaritycited by 17
  12. 2024Merging Text Transformer Models from Different Initializationscited by 16
  13. 2024Geometric Signatures of Compositionality Across a Language Model's Lifetimecited by 16
  14. 2025Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddingscited by 16
  15. 2023Outlier Dimensions Encode Task-Specific Knowledgecited by 15
  16. 2024Understanding and Minimising Outlier Features in Neural Network Trainingcited by 10
  17. 2025Metis: Training LLMs with FP4 Quantizationcited by 9
  18. 2024Outlier-weighed Layerwise Sampling for LLM Fine-tuningcited by 8
  19. 2025CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Mergingcited by 8
  20. 2025Fast and Low-Cost Genomic Foundation Models via Outlier Removalcited by 8
  21. 2025Randomly Removing 50% of Dimensions in Text Embeddings has Minimal Impact on Retrieval and Classification Taskscited by 8
  22. 2022A Natural Bias for Language Generation Modelscited by 6
  23. 2025DLP: Dynamic Layerwise Pruning in Large Language Modelscited by 6
  24. 2025TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummiescited by 6
  25. 2023Is Anisotropy Inherent to Transformers?cited by 5
  26. 2023Headless Language Models: Learning without Predicting with Contrastive Weight Tyingcited by 5
  27. 2024Improving Interpretability and Robustness for the Detection of AI-Generated Imagescited by 5
  28. 2024Efficient Pruning of Text-to-Image Models: Insights from Pruning Stable Diffusioncited by 5
  29. 2025Not a nuisance but a useful heuristic: Outlier dimensions favor frequent tokens in language modelscited by 3
  30. 2026SD-MoE: Spectral Decomposition for Effective Expert Specializationcited by 3
  31. 2025Beyond One-Size-Fits-All Pruning via Evolutionary Metric Search for Large Language Modelscited by 2
  32. 2024CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compressioncited by 1
  33. 2025On Representational Dissociation of Language and Arithmetic in Large Language Modelscited by 1
  34. 2026The Curse and Blessing of Mean Bias in FP4-Quantized LLM Trainingcited by 1
  35. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inferencecited by 1
  36. 2024How much do contextualized representations encode long-range context?cited by 0
  37. 2025MERIT: Maximum-normalized Element-wise Ratio for Language Model Large-batch Trainingcited by 0
  38. 2026Efficient Post-Training Pruning of Large Language Models with Statistical Correctioncited by 0

50 most cited of 55 All 55 (JSON)