mlsanityData to 7 Oct 2026

Perspectives on the State and Future of Deep Learning - 2023

arXiv 2312.09323 ↗First posted 7 Dec 2023cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

The goal of this series is to chronicle opinions and issues in the field of machine learning as they stand today and as they change over time. The plan is to host this survey periodically until the AI singularity paperclip-frenzy-driven doomsday, keeping an updated list of topical questions and interviewing new community members for each edition. In this issue, we probed people's opinions on interpretable AI, the value of benchmarking in modern NLP, the state of progress towards understanding deep learning, and the future of academia.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 27 of its 40 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

27 of 40 references.

References · 27

  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  3. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  6. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  7. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  8. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  9. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  10. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  11. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  12. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
15 more
  1. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  2. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  3. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 185
  4. 2019Practical Deep Learning with Bayesian Principlescited by 158
  5. 2017A Bayesian Perspective on Generalization and Stochastic Gradient Descentcited by 134
  6. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 125
  7. 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 75
  8. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 74
  9. 2021Stochastic Training is Not Necessary for Generalizationcited by 63
  10. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  11. 2019Understanding Generalization through Visualizationscited by 54
  12. 2020Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisitedcited by 50
  13. 2022How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularizationcited by 40
  14. 2021Dangers of Bayesian Model Averaging under Covariate Shiftcited by 31
  15. 2023CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebracited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2024Calibrating Large Language Models Using Their Generations Onlycited by 63
  2. 2023Better, Not Just More: Data-Centric Machine Learning for Earth Observationcited by 19
  3. 2024Spin glass model of in-context learningcited by 2
  4. 2025Gaussian Universality for Diffusion Modelscited by 0