# Two-stage Training for Learning from Label Proportions

arXiv: https://arxiv.org/abs/2105.10635 · HTML: https://mlsanity.com/paper/2105.10635/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 22 May 2021.

## Builds on (most cited first, 13 of 13)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2001.06937/) (2020)
- [On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data](https://mlsanity.com/paper/1808.10585/) (2018)
- [On Learning from Label Proportions](https://mlsanity.com/paper/1402.5902/) (2014)
- [Learning from Label Proportions with Consistency Regularization](https://mlsanity.com/paper/1910.13188/) (2019)
- [Deep multi-class learning from label proportions](https://mlsanity.com/paper/1905.12909/) (2019)
- [Co-training for Demographic Classification Using Deep Learning from Label Proportions](https://mlsanity.com/paper/1709.04108/) (2017)
- [Learning from Label Proportions with Generative Adversarial Networks](https://mlsanity.com/paper/1909.02180/) (2019)

## Built on by (most cited first, 19 of 19)

- [Learning from Label Proportions by Learning with Label Noise](https://mlsanity.com/paper/2203.02496/) (2022)
- [Learning from Label Proportion with Online Pseudo-Label Decision by Regret Minimization](https://mlsanity.com/paper/2302.08947/) (2023)
- [MixBag: Bag-Level Data Augmentation for Learning from Label Proportions](https://mlsanity.com/paper/2308.08822/) (2023)
- [Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagation](https://mlsanity.com/paper/2310.08056/) (2023)
- [Learning crop type mapping from regional label proportions in large-scale SAR and optical imagery](https://mlsanity.com/paper/2208.11607/) (2022)
- [LLP-Bench: A Large Scale Tabular Benchmark for Learning from Label Proportions](https://mlsanity.com/paper/2310.10096/) (2023)
- [Learning from Label Proportions with Instance-wise Consistency](https://mlsanity.com/paper/2203.12836/) (2022)
- [SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation](https://mlsanity.com/paper/2212.07072/) (2022)
- [Data Efficient Training with Imbalanced Label Sample Distribution for Fashion Detection](https://mlsanity.com/paper/2305.04379/) (2023)
- [Theoretical Proportion Label Perturbation for Learning from Label Proportions in Large Bags](https://mlsanity.com/paper/2408.14130/) (2024)
- [Forming Auxiliary High-confident Instance-level Loss to Promote Learning from Label Proportions](https://mlsanity.com/paper/2411.10364/) (2024)
- [Weakly-Supervised Domain Adaptation with Proportion-Constrained Pseudo-Labeling](https://mlsanity.com/paper/2506.22301/) (2025)
- [Semantic Embedded Deep Neural Network: A Generic Approach to Boost Multi-Label Image Classification Performance](https://mlsanity.com/paper/2305.05228/) (2023)
- [Learning under Label Proportions for Text Classification](https://mlsanity.com/paper/2310.11707/) (2023)
- [Evaluating LLP Methods: Challenges and Approaches](https://mlsanity.com/paper/2310.19065/) (2023)
- [FRACTAL: Fine-Grained Scoring from Aggregate Text Labels](https://mlsanity.com/paper/2404.04817/) (2024)
- [Class-aware and Augmentation-free Contrastive Learning from Label Proportion](https://mlsanity.com/paper/2408.06743/) (2024)
- [Learning from Label Proportions and Covariate-shifted Instances](https://mlsanity.com/paper/2411.12334/) (2024)
- [Learning from Label Proportions with Dual-proportion Constraints](https://mlsanity.com/paper/2603.21153/) (2026)

## Abstract (arXiv)

Learning from label proportions (LLP) aims at learning an instance-level classifier with label proportions in grouped training data. Existing deep learning based LLP methods utilize end-to-end pipelines to obtain the proportional loss with Kullback-Leibler divergence between the bag-level prior and posterior class distributions. However, the unconstrained optimization on this objective can hardly reach a solution in accordance with the given proportions. Besides, concerning the probabilistic classifier, this strategy unavoidably results in high-entropy conditional class distributions at the instance level. These issues further degrade the performance of the instance-level classification. In this paper, we regard these problems as noisy pseudo labeling, and instead impose the strict proportion consistency on the classifier with a constrained optimization as a continuous training stage for existing LLP classifiers. In addition, we introduce the mixup strategy and symmetric crossentropy to further reduce the label noise. Our framework is model-agnostic, and demonstrates compelling performance improvement in extensive experiments, when incorporated into other deep LLP models as a post-hoc phase.