# Loss factorization, weakly supervised learning and label noise robustness

arXiv: https://arxiv.org/abs/1602.02450 · HTML: https://mlsanity.com/paper/1602.02450/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2016.

## Built on by (most cited first, 24 of 24)

- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Towards Label-free Scene Understanding by Vision Foundation Models](https://mlsanity.com/paper/2306.03899/) (2023)
- [Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimation](https://mlsanity.com/paper/2206.02791/) (2022)
- [Positive-Unlabeled Reward Learning](https://mlsanity.com/paper/1911.00459/) (2019)
- [Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection](https://mlsanity.com/paper/2111.14932/) (2021)
- [Enhanced H-Consistency Bounds](https://mlsanity.com/paper/2407.13722/) (2024)
- [Towards Data-centric Graph Machine Learning: Review and Outlook](https://mlsanity.com/paper/2309.10979/) (2023)
- [Analysing the Noise Model Error for Realistic Noisy Label Data](https://mlsanity.com/paper/2101.09763/) (2021)
- [DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems](https://mlsanity.com/paper/2211.05764/) (2022)
- [Robust Nonparametric Regression under Poisoning Attack](https://mlsanity.com/paper/2305.16771/) (2023)
- [Binary classification with corrupted labels](https://mlsanity.com/paper/2106.09136/) (2021)
- [Do We Really Need Gold Samples for Sample Weighting Under Label Noise?](https://mlsanity.com/paper/2104.09045/) (2021)
- [Multi-class Label Noise Learning via Loss Decomposition and Centroid Estimation](https://mlsanity.com/paper/2203.10858/) (2022)
- [Improving Data Quality with Training Dynamics of Gradient Boosting Decision Trees](https://mlsanity.com/paper/2210.11327/) (2022)
- [Multiclass Learning from Noisy Labels for Non-decomposable Performance Measures](https://mlsanity.com/paper/2402.01055/) (2024)
- [Bias-Variance Decompositions for Margin Losses](https://mlsanity.com/paper/2204.12155/) (2022)
- [Deep Positive-Unlabeled Anomaly Detection for Contaminated Unlabeled Data](https://mlsanity.com/paper/2405.18929/) (2024)
- [Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions](https://mlsanity.com/paper/2103.13607/) (2021)
- [CSGNN: Conquering Noisy Node labels via Dynamic Class-wise Selection](https://mlsanity.com/paper/2311.11473/) (2023)
- [Robust Learning under Hybrid Noise](https://mlsanity.com/paper/2407.04029/) (2024)
- [Skill-Enhanced Reinforcement Learning Acceleration from Heterogeneous Demonstrations](https://mlsanity.com/paper/2412.06207/) (2024)
- [Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels](https://mlsanity.com/paper/2605.20347/) (2026)
- [Towards Harnessing Feature Embedding for Robust Learning with Noisy Labels](https://mlsanity.com/paper/2206.13025/) (2022)
- [NCSAM Noise-Compensated Sharpness-Aware Minimization for Noisy Label Learning](https://mlsanity.com/paper/2601.19947/) (2026)

## Abstract (arXiv)

We prove that the empirical risk of most well-known loss functions factors into a linear term aggregating all labels with a term that is label free, and can further be expressed by sums of the loss. This holds true even for non-smooth, non-convex losses and in any RKHS. The first term is a (kernel) mean operator --the focal quantity of this work-- which we characterize as the sufficient statistic for the labels. The result tightens known generalization bounds and sheds new light on their interpretation. Factorization has a direct application on weakly supervised learning. In particular, we demonstrate that algorithms like SGD and proximal methods can be adapted with minimal effort to handle weak supervision, once the mean operator has been estimated. We apply this idea to learning with asymmetric noisy labels, connecting and extending prior work. Furthermore, we show that most losses enjoy a data-dependent (by the mean operator) form of noise robustness, in contrast with known negative results.