mlsanityData to 7 Oct 2026

Prospective Prediction of OOD Degradation from Source-Side Training Dynamics

arXiv 2610.12397 ↗First posted 8 Oct 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study whether persistent out-of-distribution (OOD) degradation can be predicted before it is directly observed using only source-side training dynamics. In a controlled shortcut-learning setting, a simple logistic regression predictor develops a clear prospective signal, while training time alone does not. Temporal summaries of the source-side quantities are substantially more informative than their current values. When transferred without additional training from a CNN to an MLP, confidence and entropy dynamics retain substantial predictive information. These results provide a proof of principle that source-side training dynamics can contain an early warning signal for future OOD failure.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 16 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

15 of 16 references.

References · 15

  1. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  4. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  5. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  6. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  7. 2021Unsolved Problems in ML Safetycited by 291
  8. 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 141
  9. 2020Learning from Failure: Training Debiased Classifier from Biased Classifiercited by 133
  10. 2022Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shiftcited by 94
  11. 2022Detecting Shortcut Learning for Fair Medical AI using Shortcut Testingcited by 52
  12. 2024Shortcut Learning in Medical Image Segmentationcited by 16
3 more
  1. 2024Demonstrating and Reducing Shortcuts in Vision-Language Representation Learningcited by 14
  2. 2024Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlationscited by 12
  3. 2023Beyond Distribution Shift: Spurious Features Through the Lens of Training Dynamicscited by 11

Built on by

Not cited yet.