mlsanityData to 7 Oct 2026

Beyond the Class Subspace: Teacher-Guided Training for Reliable Out-of-Distribution Detection in Single-Domain Models

arXiv 2603.11269 ↗First posted 11 Mar 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution (OOD) detection methods perform well on multi-domain benchmarks, yet many practical systems are trained on single-domain data. We show that this regime induces a geometric failure mode, Domain-Sensitivity Collapse (DSC): supervised training compresses features into a low-rank class subspace and suppresses directions that carry domain-shift signal. We provide theory showing that, under DSC, distance- and logit-based OOD scores lose sensitivity to domain shift. We then introduce Teacher-Guided Training (TGT), which distills class-suppressed residual structure from a frozen multi-domain teacher (DINOv2) into the student during training. The teacher and auxiliary head are discarded after training, adding no inference overhead. Across eight single-domain benchmarks, TGT yields large far-OOD FPR@95 reductions for distance-based scorers: MDS improves by 11.61 pp, ViM by 10.78 pp, and kNN by 12.87 pp (ResNet-50 average), while maintaining or slightly improving in-domain OOD and classification accuracy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 38 references that are in the map. First posted on arXiv 11 Mar 2026.

Builds on

32 of 38 references.

References · 32

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  8. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  9. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  10. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  11. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  12. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
20 more
  1. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  2. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  3. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  4. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  5. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  6. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  7. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  8. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  9. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  10. 2021SSD: A Unified Framework for Self-Supervised Outlier Detectioncited by 264
  11. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 226
  12. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  13. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  14. 2021On the Role of Neural Collapse in Transfer Learningcited by 100
  15. 2023Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancementcited by 77
  16. 2020A Benchmark of Medical Out of Distribution Detectioncited by 40
  17. 2021Confidence-based Out-of-Distribution Detection: A Comparative Study and Analysiscited by 34
  18. 2023Detecting Out-of-Distribution Through the Lens of Neural Collapsecited by 33
  19. 2022Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networkscited by 23
  20. 2025Can We Ignore Labels In Out of Distribution Detection?cited by 3

Built on by

Not cited yet.