mlsanityData to 7 Oct 2026

Decoupled Entropy Minimization

arXiv 2511.03256 ↗First posted 5 Nov 2025cs.LG cs.CV cs.ITcited by 1

Not on the map. Why?

Abstract (arXiv)

Entropy Minimization (EM) is beneficial to reducing class overlap, bridging domain gap, and restricting uncertainty for various tasks in machine learning, yet its potential is limited. To study the internal mechanism of EM, we reformulate and decouple the classical EM into two parts with opposite effects: cluster aggregation driving factor (CADF) rewards dominant classes and prompts a peaked output distribution, while gradient mitigation calibrator (GMC) penalizes high-confidence classes based on predicted probabilities. Furthermore, we reveal the limitations of classical EM caused by its coupled formulation: 1) reward collapse impedes the contribution of high-certainty samples in the learning process, and 2) easy-class bias induces misalignment between output distribution and label distribution. To address these issues, we propose Adaptive Decoupled Entropy Minimization (AdaDEM), which normalizes the reward brought from CADF and employs a marginal entropy calibrator (MEC) to replace GMC. AdaDEM outperforms DEM*, an upper-bound variant of classical EM, and achieves superior performance across various imperfectly supervised learning tasks in noisy and dynamic environments.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 5 Nov 2025.

Builds on

41 of 58 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  9. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  10. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  11. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  12. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
29 more
  1. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  2. 2019Natural Adversarial Examplescited by 1,401
  3. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  4. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  5. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  6. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  8. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 844
  9. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  10. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 682
  11. 2018ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentationcited by 623
  12. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  13. 2019Confidence Regularized Self-Trainingcited by 418
  14. 2020Early-Learning Regularization Prevents Memorization of Noisy Labelscited by 414
  15. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 409
  16. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
  17. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  18. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  19. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  20. 2020Contrastive Clusteringcited by 264
  21. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  22. 2017Learning Discrete Representations via Information Maximizing Self-Augmented Trainingcited by 149
  23. 2024Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factorscited by 106
  24. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  25. 2015Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networkscited by 94
  26. 2016Mutual Exclusivity Loss for Semi-Supervised Deep Learningcited by 22
  27. 2021The Semi-Supervised iNaturalist-Aves Challenge at FGVC7 Workshopcited by 21
  28. 2025Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Groundingcited by 9
  29. 2022Aligning Logits Generatively for Principled Black-Box Knowledge Distillationcited by 3

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Neural Collapse in Test-Time Adaptationcited by 1