# Pseudo-label Learning with Calibrated Confidence Using an Energy-based Model

arXiv: https://arxiv.org/abs/2404.09585 · HTML: https://mlsanity.com/paper/2404.09585/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 24 of 24)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Improved Contrastive Divergence Training of Energy Based Models](https://mlsanity.com/paper/2012.01316/) (2020)
- [Residual Energy-Based Models for Text Generation](https://mlsanity.com/paper/2004.11714/) (2020)
- [Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration](https://mlsanity.com/paper/2109.14285/) (2021)
- [Debiased Learning from Naturally Imbalanced Pseudo-Labels](https://mlsanity.com/paper/2201.01490/) (2022)
- [BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotation](https://mlsanity.com/paper/2203.02533/) (2022)
- [Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Model](https://mlsanity.com/paper/1812.10907/) (2018)
- [Learning for Single-Shot Confidence Calibration in Deep Neural Networks through Stochastic Inferences](https://mlsanity.com/paper/1809.10877/) (2018)
- [EDropout: Energy-Based Dropout and Pruning of Deep Neural Networks](https://mlsanity.com/paper/2006.04270/) (2020)
- [Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruning](https://mlsanity.com/paper/2212.10005/) (2022)
- [Energy-Based Continuous Inverse Optimal Control](https://mlsanity.com/paper/1904.05453/) (2019)
- [Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection](https://mlsanity.com/paper/2303.14404/) (2023)
- [A Hybrid of Generative and Discriminative Models Based on the Gaussian-coupled Softmax Layer](https://mlsanity.com/paper/2305.05912/) (2023)

## Abstract (arXiv)

In pseudo-labeling (PL), which is a type of semi-supervised learning, pseudo-labels are assigned based on the confidence scores provided by the classifier; therefore, accurate confidence is important for successful PL. In this study, we propose a PL algorithm based on an energy-based model (EBM), which is referred to as the energy-based PL (EBPL). In EBPL, a neural network-based classifier and an EBM are jointly trained by sharing their feature extraction parts. This approach enables the model to learn both the class decision boundary and input data distribution, enhancing confidence calibration during network training. The experimental results demonstrate that EBPL outperforms the existing PL method in semi-supervised image classification tasks, with superior confidence calibration error and recognition accuracy.