mlsanityData to 7 Oct 2026

Predicting on the Edge: Identifying Where a Larger Model Does Better

arXiv 2202.07652 ↗First posted 15 Feb 2022cs.LG cs.AI stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Much effort has been devoted to making large and more accurate models, but relatively little has been put into understanding which examples are benefiting from the added complexity. In this paper, we demonstrate and analyze the surprisingly tight link between a model's predictive uncertainty on individual examples and the likelihood that larger models will improve prediction on them. Through extensive numerical studies on the T5 encoder-decoder architecture, we show that large models have the largest improvement on examples where the small model is most uncertain. On more certain examples, even those where the small model is not particularly accurate, large models are often unable to improve at all, and can even perform worse than the smaller model. Based on these findings, we show that a switcher model which defers examples to a larger model when a small model is uncertain can achieve striking improvements in performance and resource usage. We also explore committee-based uncertainty metrics that can be more effective but less practical.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

31 of 48 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  9. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  10. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  11. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  12. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
19 more
  1. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
  2. 2020Calibration of Pre-trained Transformerscited by 307
  3. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  4. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  5. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  6. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  7. 2021Deep Learning Through the Lens of Example Difficultycited by 139
  8. 2018Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goalscited by 103
  9. 2020Characterizing Structural Regularities of Labeled Data in Overparameterized Modelscited by 93
  10. 2021The MultiBERTs: BERT Reproductions for Robustness Analysiscited by 89
  11. 2021Active Testing: Sample-Efficient Model Evaluationcited by 82
  12. 2017IDK Cascades: Fast Deep Learning by Learning not to Overthinkcited by 47
  13. 2021Supervised Transfer Learning at Scale for Medical Imagingcited by 38
  14. 2021Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Levelcited by 34
  15. 2021On the Reproducibility of Neural Network Predictionscited by 28
  16. 2021Churn Reduction via Distillationcited by 17
  17. 2017Adaptive Classification for Prediction Under a Budgetcited by 15
  18. 2021When in Doubt, Summon the Titans: Efficient Inference with Large Modelscited by 8
  19. 2021Locally Adaptive Label Smoothing for Predictive Churncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 1; 2025: 0; 2026: 1.

6 papers.

  1. 2023When Does Confidence-Based Cascade Deferral Suffice?cited by 58
  2. 2023ERSAM: Neural Architecture Search For Energy-Efficient and Real-Time Social Ambiance Measurementcited by 1
  3. 2023NetDistiller: Empowering Tiny Deep Learning via In-Situ Distillationcited by 0
  4. 2023Streaming Lossless Volumetric Compression of Medical Images Using Gated Recurrent Convolutional Neural Networkcited by 0
  5. 2024Dual-Model Distillation for Efficient Action Classification with Hybrid Edge-Cloud Solutioncited by 0
  6. 2026Inducing Artificial Uncertainty in Language Modelscited by 0