# Semi-Supervised Deep Regression with Uncertainty Consistency and Variational Model Ensembling via Bayesian Neural Networks

arXiv: https://arxiv.org/abs/2302.07579 · HTML: https://mlsanity.com/paper/2302.07579/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 15 Feb 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/1907.07034/) (2019)
- [Semi-Supervised Semantic Segmentation with Cross-Consistency Training](https://mlsanity.com/paper/2003.09005/) (2020)
- [Transformation Consistent Self-ensembling Model for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/1903.00348/) (2019)
- [AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations rather than Data](https://mlsanity.com/paper/1901.04596/) (2019)
- [3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training](https://mlsanity.com/paper/1811.12506/) (2018)
- [Enhancing Pseudo Label Quality for Semi-Supervised Domain-Generalized Medical Image Segmentation](https://mlsanity.com/paper/2201.08657/) (2022)
- [Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2105.07059/) (2021)
- [Semi-supervised Skin Lesion Segmentation via Transformation Consistent Self-ensembling Model](https://mlsanity.com/paper/1808.03887/) (2018)
- [Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance](https://mlsanity.com/paper/1805.10407/) (2018)
- [Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition](https://mlsanity.com/paper/2112.09690/) (2021)
- [Calibrating Label Distribution for Class-Imbalanced Barely-Supervised Knee Segmentation](https://mlsanity.com/paper/2205.03644/) (2022)
- [Deep Ordinal Regression with Label Diversity](https://mlsanity.com/paper/2006.15864/) (2020)
- [Adaptive Contrast for Image Regression in Computer-Aided Disease Assessment](https://mlsanity.com/paper/2112.11700/) (2021)
- [FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filtering](https://mlsanity.com/paper/2203.15765/) (2022)
- [Semi-supervised FusedGAN for Conditional Image Generation](https://mlsanity.com/paper/1801.05551/) (2018)
- [Cyclical Self-Supervision for Semi-Supervised Ejection Fraction Prediction from Echocardiogram Videos](https://mlsanity.com/paper/2210.11291/) (2022)
- [Deep Kernels with Probabilistic Embeddings for Small-Data Learning](https://mlsanity.com/paper/1910.05858/) (2019)
- [Twin Neural Network Regression is a Semi-Supervised Regression Algorithm](https://mlsanity.com/paper/2106.06124/) (2021)

## Built on by (most cited first, 16 of 16)

- [Teach CLIP to Develop a Number Sense for Ordinal Regression](https://mlsanity.com/paper/2408.03574/) (2024)
- [Urban Safety Perception Assessments via Integrating Multimodal Large Language Models with Street View Images](https://mlsanity.com/paper/2407.19719/) (2024)
- [RankUp: Boosting Semi-Supervised Regression with an Auxiliary Ranking Classifier](https://mlsanity.com/paper/2410.22124/) (2024)
- [Taste More, Taste Better: Diverse Data and Strong Model Boost Semi-Supervised Crowd Counting](https://mlsanity.com/paper/2503.17984/) (2025)
- [Semi-supervised Node Importance Estimation with Informative Distribution Modeling for Uncertainty Regularization](https://mlsanity.com/paper/2503.20697/) (2025)
- [Bayesian Self-Training for Semi-Supervised 3D Segmentation](https://mlsanity.com/paper/2409.08102/) (2024)
- [CARE: Confidence-Aware Regression Estimation of building density fine-tuning EO Foundation Models](https://mlsanity.com/paper/2502.13734/) (2025)
- [UPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression](https://mlsanity.com/paper/2508.03520/) (2025)
- [Bridged Clustering: Semi-Supervised Sparse Bridging](https://mlsanity.com/paper/2510.07182/) (2025)
- [Ranking-aware Reinforcement Learning for Ordinal Ranking](https://mlsanity.com/paper/2601.20585/) (2026)
- [Toward Robust Semi-supervised Regression via Dual-stream Knowledge Distillation](https://mlsanity.com/paper/2508.14082/) (2025)
- [Contrastive Learning for Semi-Supervised Deep Regression with Generalized Ordinal Rankings from Spectral Seriation](https://mlsanity.com/paper/2512.09267/) (2025)
- [Semi-Supervised Gaze Estimation via Disentangled Subspace Contrastive Learning](https://mlsanity.com/paper/2605.27080/) (2026)
- [WhiteCon: Semi-Supervised Domain Adaptation Regression Through Whitening Transform and Dual Consistency](https://mlsanity.com/paper/2609.34078/) (2026)
- [SpecRegMatch: Robust Semi-Supervised Regression for Vehicle Interior Noise Prediction](https://mlsanity.com/paper/2609.34111/) (2026)
- [CoGameReg: Cooperative-Game-Based Collaborative Semi-Supervised Regression for Seismic Reservoir Property Prediction](https://mlsanity.com/paper/2610.12884/) (2026)

## Abstract (arXiv)

Deep regression is an important problem with numerous applications. These range from computer vision tasks such as age estimation from photographs, to medical tasks such as ejection fraction estimation from echocardiograms for disease tracking. Semi-supervised approaches for deep regression are notably under-explored compared to classification and segmentation tasks, however. Unlike classification tasks, which rely on thresholding functions for generating class pseudo-labels, regression tasks use real number target predictions directly as pseudo-labels, making them more sensitive to prediction quality. In this work, we propose a novel approach to semi-supervised regression, namely Uncertainty-Consistent Variational Model Ensembling (UCVME), which improves training by generating high-quality pseudo-labels and uncertainty estimates for heteroscedastic regression. Given that aleatoric uncertainty is only dependent on input data by definition and should be equal for the same inputs, we present a novel uncertainty consistency loss for co-trained models. Our consistency loss significantly improves uncertainty estimates and allows higher quality pseudo-labels to be assigned greater importance under heteroscedastic regression. Furthermore, we introduce a novel variational model ensembling approach to reduce prediction noise and generate more robust pseudo-labels. We analytically show our method generates higher quality targets for unlabeled data and further improves training. Experiments show that our method outperforms state-of-the-art alternatives on different tasks and can be competitive with supervised methods that use full labels. Our code is available at https://github.com/xmed-lab/UCVME.