# Learning under Label Proportions for Text Classification

arXiv: https://arxiv.org/abs/2310.11707 · HTML: https://mlsanity.com/paper/2310.11707/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 18 Oct 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts](https://mlsanity.com/paper/1307.5336/) (2013)
- [Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization](https://mlsanity.com/paper/2102.02414/) (2021)
- [Learning Longer-term Dependencies in RNNs with Auxiliary Losses](https://mlsanity.com/paper/1803.00144/) (2018)
- [Effective Transfer Learning for Identifying Similar Questions: Matching User Questions to COVID-19 FAQs](https://mlsanity.com/paper/2008.13546/) (2020)
- [Learning from Label Proportions with Consistency Regularization](https://mlsanity.com/paper/1910.13188/) (2019)
- [Deep multi-class learning from label proportions](https://mlsanity.com/paper/1905.12909/) (2019)
- [Co-training for Demographic Classification Using Deep Learning from Label Proportions](https://mlsanity.com/paper/1709.04108/) (2017)
- [Learning from Label Proportions with Generative Adversarial Networks](https://mlsanity.com/paper/1909.02180/) (2019)
- [Two-stage Training for Learning from Label Proportions](https://mlsanity.com/paper/2105.10635/) (2021)
- [Challenges and approaches to privacy preserving post-click conversion prediction](https://mlsanity.com/paper/2201.12666/) (2022)
- [Learning from Label Proportions: A Mutual Contamination Framework](https://mlsanity.com/paper/2006.07330/) (2020)
- [Robust Bayesian Inference for Discrete Outcomes with the Total Variation Distance](https://mlsanity.com/paper/2010.13456/) (2020)
- [Auxiliary Objectives for Neural Error Detection Models](https://mlsanity.com/paper/1707.05227/) (2017)
- [On the Complexity of Learning from Label Proportions](https://mlsanity.com/paper/2004.03515/) (2020)
- [Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks](https://mlsanity.com/paper/2107.07610/) (2021)

## Built on by (most cited first, 1 of 1)

- [FRACTAL: Fine-Grained Scoring from Aggregate Text Labels](https://mlsanity.com/paper/2404.04817/) (2024)

## Abstract (arXiv)

We present one of the preliminary NLP works under the challenging setup of Learning from Label Proportions (LLP), where the data is provided in an aggregate form called bags and only the proportion of samples in each class as the ground truth. This setup is inline with the desired characteristics of training models under Privacy settings and Weakly supervision. By characterizing some irregularities of the most widely used baseline technique DLLP, we propose a novel formulation that is also robust. This is accompanied with a learnability result that provides a generalization bound under LLP. Combining this formulation with a self-supervised objective, our method achieves better results as compared to the baselines in almost 87% of the experimental configurations which include large scale models for both long and short range texts across multiple metrics.