mlsanityData to 7 Oct 2026

Positive-Congruent Training: Towards Regression-Free Model Updates

arXiv 2011.09161 ↗First posted 18 Nov 2020cs.CV cs.LGcited by 55

Not on the map. Why?

Abstract (arXiv)

Reducing inconsistencies in the behavior of different versions of an AI system can be as important in practice as reducing its overall error. In image classification, sample-wise inconsistencies appear as "negative flips": A new model incorrectly predicts the output for a test sample that was correctly classified by the old (reference) model. Positive-congruent (PC) training aims at reducing error rate while at the same time reducing negative flips, thus maximizing congruency with the reference model only on positive predictions, unlike model distillation. We propose a simple approach for PC training, Focal Distillation, which enforces congruence with the reference model by giving more weights to samples that were correctly classified. We also found that, if the reference model itself can be chosen as an ensemble of multiple deep neural networks, negative flips can be further reduced without affecting the new model's accuracy.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 12; 2023: 6; 2024: 11; 2025: 7; 2026: 14.

55 papers.

Sort:

By area:

  1. 2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 62
  2. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  3. 2022Monitored Distillation for Positive Congruent Depth Completioncited by 24
  4. 2024Predictive Churn with the Set of Good Modelscited by 15
  5. 2024MUSCLE: A Model Update Strategy for Compatible LLM Evolutioncited by 15
  6. 2021Backward-Compatible Prediction Updates: A Probabilistic Approachcited by 14
  7. 2021CoReS: Compatible Representations via Stationaritycited by 14
  8. 2024Good Teachers Explain: Explanation-Enhanced Knowledge Distillationcited by 14
  9. 2021Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updatescited by 13
  10. 2022Measuring and Reducing Model Update Regression in Structured Prediction for NLPcited by 11
  11. 2023AI Model Disgorgement: Methods and Choicescited by 9
  12. 2022NeuRecover: Regression-Controlled Repair of Deep Neural Networks with Training Historycited by 8
38 more
  1. 2025boldsymbolλ-Orthogonality Regularization for Compatible Representation Learningcited by 8
  2. 2022Multi-Modal Learning Using Physicians Diagnostics for Optical Coherence Tomography Classificationcited by 7
  3. 2022ELODI: Ensemble Logit Difference Inhibition for Positive-Congruent Trainingcited by 7
  4. 2023Backward Compatibility During Data Updates by Weight Interpolationcited by 7
  5. 2023FastFill: Efficient Compatible Model Updatecited by 7
  6. 2022Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrievalcited by 6
  7. 2023Improving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusioncited by 6
  8. 2021Fixes That Fail: Self-Defeating Improvements in Machine-Learning Systemscited by 5
  9. 2024Meanings and Feelings of Large Language Models: Observability of Latent States in Generative AIcited by 5
  10. 2024UnCLe: Benchmarking Unsupervised Continual Learning for Depth Completioncited by 5
  11. 2022Darwinian Model Upgrades: Model Evolving with Selective Compatibilitycited by 4
  12. 2024Robustness-Congruent Adversarial Training for Secure Machine Learning Model Updatescited by 4
  13. 2024Towards Cross-modal Backward-compatible Representation Learning for Vision-Language Modelscited by 4
  14. 2025Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learningcited by 4
  15. 2026Beyond the Mean: Within-Model Reliable Change Detection for LLM Evaluationcited by 4
  16. 2023Clinical Trial Active Learningcited by 3
  17. 2024Backward-Compatible Aligned Representations via an Orthogonal Transformation Layercited by 3
  18. 2025Mitigating Negative Flips via Margin Preserving Trainingcited by 3
  19. 2021Distributionally Robust Group Backwards Compatibilitycited by 2
  20. 2024FlipGuard: Defending Preference Alignment against Update Regression with Constrained Optimizationcited by 2
  21. 2025Regression-aware Continual Learning for Android Malware Detectioncited by 2
  22. 2025On the Risk of Misleading Reports: Diagnosing Textual Biases in Multimodal Clinical AIcited by 2
  23. 2026Pay Only for Disagreement: Certified No-Regression Verdicts for Model Updates with Matching Label-Complexity Boundscited by 2
  24. 2022Contrastive Neighborhood Alignmentcited by 1
  25. 2022Switchable Representation Learning Framework with Self-compatibilitycited by 1
  26. 2023Maintaining Stability and Plasticity for Predictive Churn Reductioncited by 1
  27. 2024Backward Compatibility in Attributive Explanation and Enhanced Model Training Methodcited by 1
  28. 2024Targeting Negative Flips in Active Learning using Validation Setscited by 1
  29. 2025An Experience Report on Regression-Free Repair of Deep Neural Network Modelcited by 1
  30. 2026Medical Context Distorts Decisions in Clinical Vision Language Modelscited by 1
  31. 2022Towards Regression-Free Neural Networks for Diverse Compute Platformscited by 0
  32. 2025DeepTrust: Multi-Step Classification through Dissimilar Adversarial Representations for Robust Android Malware Detectioncited by 0
  33. 2026Regression Test Selection for Updated Capability Modules in Compositional ML Systems via Atomic-Quality Probescited by 0
  34. 2026Lottery Tickets Are Not Deployment Ticketscited by 0
  35. 2026No Universal Signal Predicts Sample-Level LLM Regression under Version Updatescited by 0
  36. 2026UpgradeBench: A Decision-Centric Benchmark for Upgrading Fine-Tuned LLM Specialistscited by 0
  37. 2026Rethinking the Teacher-Student Framework for Test-Time Adaptationcited by 0
  38. 2026Faster Block-Diffusion Serving with Distribution-Free Risk Guaranteescited by 0

50 most cited of 55 All 55 (JSON)