mlsanityData to 7 Oct 2026

The Fair Language Model Paradox

arXiv 2410.11985 ↗First posted 15 Oct 2024cs.CL cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are widely deployed in real-world applications, yet little is known about their training dynamics at the token level. Evaluation typically relies on aggregated training loss, measured at the batch level, which overlooks subtle per-token biases arising from (i) varying token-level dynamics and (ii) structural biases introduced by hyperparameters. While weight decay is commonly used to stabilize training, we reveal that it silently introduces performance biases detectable only at the token level. In fact, we empirically show across different dataset sizes, model architectures and sizes ranging from 270M to 3B parameters that as weight decay increases, low-frequency tokens are disproportionately depreciated. This is particularly concerning, as these neglected low-frequency tokens represent the vast majority of the token distribution in most languages, calling for novel regularization techniques that ensure fairness across all available tokens.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 45 of its 63 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

45 of 63 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  6. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  7. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  8. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  9. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  10. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  11. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  12. 2018Stronger generalization bounds for deep nets via a compression approachcited by 306
33 more
  1. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  2. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  3. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  4. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  5. 2017L2 Regularization versus Batch and Weight Normalizationcited by 121
  6. 2019An Exponential Learning Rate Schedule for Deep Learningcited by 121
  7. 2024Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabulariescited by 113
  8. 2018Three Mechanisms of Weight Decay Regularizationcited by 112
  9. 2020Optimal Regularization Can Mitigate Double Descentcited by 107
  10. 2023Explaining grokking through circuit efficiencycited by 105
  11. 2021On the Role of Neural Collapse in Transfer Learningcited by 100
  12. 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
  13. 2020Identifying and Compensating for Feature Deviation in Imbalanced Deep Learningcited by 76
  14. 2024OpenELM: An Efficient Language Model Family with Open Training and Inference Frameworkcited by 74
  15. 2019Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentationcited by 71
  16. 2023Why Do We Need Weight Decay in Modern Deep Learning?cited by 70
  17. 2022The Effects of Regularization and Data Augmentation are Class Dependentcited by 67
  18. 2022Impact of Tokenization on Language Models: An Analysis for Turkishcited by 62
  19. 2020Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Ratecited by 59
  20. 2019Adjusting Decision Boundary for Class Imbalanced Learningcited by 52
  21. 2017Compression-aware Training of Deep Networkscited by 46
  22. 2023Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Datacited by 40
  23. 2020On the training dynamics of deep networks with L2 regularizationcited by 38
  24. 2023Neural Collapse for Unconstrained Feature Model under Cross-entropy Loss with Imbalanced Datacited by 38
  25. 2023Inducing Neural Collapse in Deep Long-tailed Learningcited by 28
  26. 2018On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyondcited by 24
  27. 2021Multi-scale Feature Learning Dynamics: Insights for Double Descentcited by 24
  28. 2024Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Modelcited by 19
  29. 2022SGD and Weight Decay Secretly Minimize the Rank of Your Neural Networkcited by 17
  30. 2024Large Vocabulary Size Improves Large Language Modelscited by 11
  31. 2023Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decaycited by 9
  32. 2024How Neural Networks Learn the Support is an Implicit Regularization Effect of SGDcited by 8
  33. 2022Generalization Bounds for Few-Shot Transfer Learning with Pretrained Classifierscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Exploiting Vocabulary Frequency Imbalance in Language Model Pre-trainingcited by 7
  2. 2025A Lightweight Method to Disrupt Memorized Sequences in LLMcited by 1
  3. 2026How Confident Is the First Token? An Uncertainty-Calibrated Prompt Optimization Framework for Large Language Model Classification and Understandingcited by 1