# Theoretical Analysis of Measure Consistency Regularization for Partially Observed Data

arXiv: https://arxiv.org/abs/2602.01437 · HTML: https://mlsanity.com/paper/2602.01437/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 73 references that are in the map.
- First posted on arXiv 1 Feb 2026.

## Builds on (most cited first, 30 of 30)

- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [Generalization and Equilibrium in Generative Adversarial Nets (GANs)](https://mlsanity.com/paper/1703.00573/) (2017)
- [Missing Data Imputation using Optimal Transport](https://mlsanity.com/paper/2002.03860/) (2020)
- [Image inpainting: A review](https://mlsanity.com/paper/1909.06399/) (2019)
- [Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity](https://mlsanity.com/paper/1810.07770/) (2018)
- [Maximum Density Divergence for Domain Adaptation](https://mlsanity.com/paper/2004.12615/) (2020)
- [Robustness to Adversarial Perturbations in Learning from Incomplete Data](https://mlsanity.com/paper/1905.13021/) (2019)
- [On the Discrimination-Generalization Tradeoff in GANs](https://mlsanity.com/paper/1711.02771/) (2017)
- [Processing of missing data by neural networks](https://mlsanity.com/paper/1805.07405/) (2018)
- [Transformed Distribution Matching for Missing Value Imputation](https://mlsanity.com/paper/2302.10363/) (2023)
- [Semi-Supervised Learning by Augmented Distribution Alignment](https://mlsanity.com/paper/1905.08171/) (2019)
- [Generalization Bounds for Domain Adaptation](https://mlsanity.com/paper/1304.1574/) (2013)
- [Learning from data with structured missingness](https://mlsanity.com/paper/2304.01429/) (2023)
- [FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning](https://mlsanity.com/paper/2310.16412/) (2023)
- [Learning Multimodal VAEs through Mutual Supervision](https://mlsanity.com/paper/2106.12570/) (2021)
- [Interpolation, Approximation and Controllability of Deep Neural Networks](https://mlsanity.com/paper/2309.06015/) (2023)
- [RedCore: Relative Advantage Aware Cross-modal Representation Learning for Missing Modalities with Imbalanced Missing Rates](https://mlsanity.com/paper/2312.10386/) (2023)
- [How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric View](https://mlsanity.com/paper/1712.08244/) (2017)
- [Minimax Estimation of Neural Net Distance](https://mlsanity.com/paper/1811.01054/) (2018)
- [Semi-supervised Vector-valued Learning: Improved Bounds and Algorithms](https://mlsanity.com/paper/1909.04883/) (2019)
- [On Duality Gap as a Measure for Monitoring GAN Training](https://mlsanity.com/paper/2012.06723/) (2020)
- [On Characterizing GAN Convergence Through Proximal Duality Gap](https://mlsanity.com/paper/2105.04801/) (2021)
- [Collaboratively Learning Linear Models with Structured Missing Data](https://mlsanity.com/paper/2307.11947/) (2023)

## Abstract (arXiv)

The problem of corrupted data, missing features, or missing modalities continues to plague the modern machine learning landscape. To address this issue, a class of regularization methods that enforce consistency between imputed and fully observed data has emerged as a promising approach for improving model generalization, particularly in partially observed settings. We refer to this class of methods as Measure Consistency Regularization (MCR). Despite its empirical success in various applications, such as image inpainting, data imputation and semi-supervised learning, a fundamental understanding of the theoretical underpinnings of MCR remains limited. This paper bridges this gap by offering theoretical insights into when MCR yields a more favorable finite-sample estimation-error upper bound, viewed through the lens of neural network distance. Under ideal interpolation and compatibility conditions, we show that the MCR estimation-error upper bound is no larger than vanilla supervised training and becomes strictly smaller when the mixed-sample alternative is favorable. We then extend the analysis to the non-ideal regime, where optimization and compatibility residuals can potentially offset this finite-sample advantage. Guided by these insights, we propose a novel practical diagnostic that leverages the duality gap and a calibrated drift estimator to infer the potential benefit of MCR training. We present detailed empirical evidence to support our theoretical claims and to show the effectiveness and accuracy of our practical diagnostic. We further provide simulations on real-world datasets to show the versatility of MCR under different model architectures designed for different data sources.