mlsanityData to 14 Oct 2026

NeurIPS 2020 Competition: Predicting Generalization in Deep Learning

arXiv 2012.07976 ↗First posted 14 Dec 2020cs.LG stat.MLcited by 41

Not on the map. Why?

Abstract (arXiv)

Understanding generalization in deep learning is arguably one of the most important questions in deep learning. Deep learning has been successfully adopted to a large number of problems ranging from pattern recognition to complex decision making, but many recent researchers have raised many concerns about deep learning, among which the most important is generalization. Despite numerous attempts, conventional statistical learning approaches have yet been able to provide a satisfactory explanation on why deep learning works. A recent line of works aims to address the problem by trying to predict the generalization performance through complexity measures. In this competition, we invite the community to propose complexity measures that can accurately predict generalization of models. A robust and general complexity measure would potentially lead to a better understanding of deep learning's underlying mechanism and behavior of deep models on unseen data, or shed light on better generalization bounds. All these outcomes will be important for making deep learning more robust and reliable.

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 10; 2023: 4; 2024: 6; 2025: 3; 2026: 2.

41 papers.

Sort:

By area:

  1. 2021Assessing Generalization of SGD via Disagreementcited by 117
  2. 2021Intrinsic Dimension, Persistent Homology and Generalization in Neural Networkscited by 70
  3. 2021A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learningcited by 57
  4. 2024Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budgetcited by 30
  5. 2021Measuring Generalization with Optimal Transportcited by 25
  6. 2022How Much More Data Do I Need? Estimating Requirements for Downstream Taskscited by 23
  7. 2021Understanding the Generalization Benefit of Model Invariance from a Data Perspectivecited by 22
  8. 2022Non-Vacuous Generalisation Bounds for Shallow Neural Networkscited by 22
  9. 2021Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metricscited by 19
  10. 2021On Margins and Derandomisation in PAC-Bayescited by 19
  11. 2022Evaluating natural language processing models with generalization metrics that do not need access to any training or testing datacited by 19
  12. 2022Optimizing Data Collection for Machine Learningcited by 15
29 more
  1. 2021Predicting Deep Neural Network Generalization with Perturbation Response Curvescited by 14
  2. 2022Predicting the generalization gap in neural networks using topological data analysiscited by 9
  3. 2023Topological Data Analysis for Neural Network Analysis: A Comprehensive Surveycited by 9
  4. 2024Understanding Generalization in Quantum Machine Learning with Marginscited by 7
  5. 2021On Predicting Generalization using GANscited by 6
  6. 2022Weight Expansion: A New Perspective on Dropout and Generalizationcited by 6
  7. 2024What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?cited by 6
  8. 2022Gaussian Process Surrogate Models for Neural Networkscited by 5
  9. 2021Auditing Robot Learning for Safety and Compliance during Deploymentcited by 4
  10. 2022Predicting Out-of-Domain Generalization with Neighborhood Invariancecited by 4
  11. 2023Input margins can predict generalization toocited by 4
  12. 2024Slicing Mutual Information Generalization Bounds for Neural Networkscited by 4
  13. 2021Gi and Pal Scores: Deep Neural Network Generalization Statisticscited by 3
  14. 2022Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rulescited by 3
  15. 2025A Model Zoo on Phase Transitions in Neural Networkscited by 3
  16. 2021Evaluation of Complexity Measures for Deep Learning Generalization in Medical Image Analysiscited by 2
  17. 2021Exploring the Common Principal Subspace of Deep Features in Neural Networkscited by 2
  18. 2021In Search of Probeable Generalization Measurescited by 2
  19. 2022Performance Deterioration of Deep Learning Models after Clinical Deployment: A Case Study with Auto-segmentation for Definitive Prostate Cancer Radiotherapycited by 2
  20. 2025Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoringcited by 2
  21. 2021Practical Assessment of Generalization Performance Robustness for Deep Networks via Contrastive Examplescited by 1
  22. 2023Fantastic DNN Classifiers and How to Identify them without Datacited by 1
  23. 2024A practical generalization metric for deep networks benchmarkingcited by 1
  24. 2026Failure Modes for Deep Learning-Based Online Mapping: How to Measure and Address Themcited by 1
  25. 2021Explaining generalization in deep learning: progress and fundamental limitscited by 0
  26. 2023Decorrelating neurons using persistencecited by 0
  27. 2024Is network fragmentation a useful complexity measure?cited by 0
  28. 2025Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenoncited by 0
  29. 2026Transductive Generalization via Optimal Transport and Its Application to Graph Node Classificationcited by 0