mlsanityData to 7 Oct 2026

BERT Busters: Outlier Dimensions that Disrupt Transformers

arXiv 2105.06990 ↗First posted 14 May 2021cs.CLcited by 110

Not on the map. Why?

Abstract (arXiv)

Multiple studies have shown that Transformers are remarkably robust to pruning. Contrary to this received wisdom, we demonstrate that pre-trained Transformer encoders are surprisingly fragile to the removal of a very small number of features in the layer outputs (<0.0001% of model weights). In case of BERT and other pre-trained encoder Transformers, the affected component is the scaling factors and biases in the LayerNorm. The outliers are high-magnitude normalization parameters that emerge early in pre-training and show up consistently in the same dimensional position throughout the model. We show that disabling them significantly degrades both the MLM loss and the downstream task performance. This effect is observed across several BERT-family models and other popular pre-trained Transformer architectures, including BART, XLNet and ELECTRA; we also show a similar effect in GPT-2.

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 14 May 2021.

Builds on

29 of 36 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2016Layer Normalizationcited by 5,579
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2016Pointer Sentinel Mixture Modelscited by 2,992
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  11. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  12. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
17 more
  1. 2015Teaching Machines to Read and Comprehendcited by 1,039
  2. 2020On Layer Normalization in the Transformer Architecturecited by 906
  3. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  4. 2019Are Sixteen Heads Really Better than One?cited by 803
  5. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  6. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  7. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  8. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  9. 2019Revealing the Dark Secrets of BERTcited by 300
  10. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  11. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  12. 2019Understanding and Improving Layer Normalizationcited by 213
  13. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 184
  14. 2019Transformers without Tears: Improving the Normalization of Self-Attentioncited by 153
  15. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  16. 2020Compressing Large-Scale Transformer-Based Models: A Case Study on BERTcited by 95
  17. 2020Positional Artefacts Propagate Through Masked Language Model Embeddingscited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 11; 2023: 20; 2024: 27; 2025: 27; 2026: 22.

110 papers.

Sort:

By area:

  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  3. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  4. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  5. 2024Massive Activations in Large Language Modelscited by 277
  6. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  7. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  8. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  9. 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 163
  10. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
  11. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
  12. 2023PB-LLM: Partially Binarized Large Language Modelscited by 80
38 more
  1. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  2. 2022Measuring the Mixing of Contextual Information in the Transformercited by 63
  3. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 62
  4. 2022Outliers Dimensions that Disrupt Transformers Are Driven by Frequencycited by 55
  5. 2024The Super Weight in Large Language Modelscited by 51
  6. 2022GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformerscited by 44
  7. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  8. 2024AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Modelscited by 35
  9. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 34
  10. 2023Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Mapscited by 33
  11. 2024Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantizationcited by 33
  12. 2023VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformerscited by 32
  13. 2025Systematic Outliers in Large Language Modelscited by 32
  14. 2021An Isotropy Analysis in the Multilingual BERT Embedding Spacecited by 25
  15. 2022Neural reality of argument structure constructionscited by 25
  16. 2024Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularizationcited by 23
  17. 2026The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinkscited by 23
  18. 2024Unveiling Linguistic Regions in Large Language Modelscited by 22
  19. 2024Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMscited by 21
  20. 2026A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Trainingcited by 21
  21. 2023Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Modelscited by 19
  22. 2024OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decompositioncited by 19
  23. 2024Robust AI-Generated Text Detection by Restricted Embeddingscited by 18
  24. 2023Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similaritycited by 17
  25. 2023Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learningcited by 16
  26. 2025Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddingscited by 16
  27. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  28. 2023Outlier Dimensions Encode Task-Specific Knowledgecited by 15
  29. 2025Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probingcited by 14
  30. 2025Accurate INT8 Training Through Dynamic Block-Level Fallbackcited by 14
  31. 2023VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Corescited by 13
  32. 2024RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantizationcited by 13
  33. 2025Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Modelscited by 13
  34. 2023Stable Anisotropic Regularizationcited by 12
  35. 2024Can Large Language Models Understand Molecules?cited by 10
  36. 2024Understanding and Minimising Outlier Features in Neural Network Trainingcited by 10
  37. 2024Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Modelscited by 9
  38. 2022Isotropic Representation Can Improve Dense Retrievalcited by 8

50 most cited of 110 All 110 (JSON)