# Learning Soft Labels via Meta Learning

arXiv: https://arxiv.org/abs/2009.09496 · HTML: https://mlsanity.com/paper/2009.09496/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2020.

## Built on by (most cited first, 27 of 27)

- [Faster Meta Update Strategy for Noise-Robust Deep Learning](https://mlsanity.com/paper/2104.15092/) (2021)
- [Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive](https://mlsanity.com/paper/2301.12534/) (2023)
- [Generalized Data Weighting via Class-level Gradient Manipulation](https://mlsanity.com/paper/2111.00056/) (2021)
- [Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Space](https://mlsanity.com/paper/2206.01106/) (2022)
- [Addressing out-of-distribution label noise in webly-labelled data](https://mlsanity.com/paper/2110.13699/) (2021)
- [FORML: Learning to Reweight Data for Fairness](https://mlsanity.com/paper/2202.01719/) (2022)
- [Ambiguous Images With Human Judgments for Robust Visual Event Classification](https://mlsanity.com/paper/2210.03102/) (2022)
- [Is your noise correction noisy? PLS: Robustness to label noise with two stage detection](https://mlsanity.com/paper/2210.04578/) (2022)
- [Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes](https://mlsanity.com/paper/2412.13573/) (2024)
- [GIFT: Unlocking Full Potential of Labels in Distilled Dataset at Near-zero Cost](https://mlsanity.com/paper/2405.14736/) (2024)
- [Ask and Remember: A Questions-Only Replay Strategy for Continual Visual Question Answering](https://mlsanity.com/paper/2502.04469/) (2025)
- [Embedding contrastive unsupervised features to cluster in- and out-of-distribution noise in corrupted image datasets](https://mlsanity.com/paper/2207.01573/) (2022)
- [Semi-supervised dry herbage mass estimation using automatic data and synthetic images](https://mlsanity.com/paper/2110.13719/) (2021)
- [Adaptive Mixing of Auxiliary Losses in Supervised Learning](https://mlsanity.com/paper/2202.03250/) (2022)
- [Instance-Conditional Timescales of Decay for Non-Stationary Learning](https://mlsanity.com/paper/2212.05908/) (2022)
- [The Power of Few: Accelerating and Enhancing Data Reweighting with Coreset Selection](https://mlsanity.com/paper/2403.12166/) (2024)
- [Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions](https://mlsanity.com/paper/2103.13607/) (2021)
- [Maximising the Utility of Validation Sets for Imbalanced Noisy-label Meta-learning](https://mlsanity.com/paper/2208.08132/) (2022)
- [Label Noise: Correcting the Forward-Correction](https://mlsanity.com/paper/2307.13100/) (2023)
- [Reproducibility study of "LICO: Explainable Models with Language-Image Consistency"](https://mlsanity.com/paper/2410.13989/) (2024)
- [GovRelBench:A Benchmark for Government Domain Relevance](https://mlsanity.com/paper/2507.21419/) (2025)
- [SNCE: Geometry-Aware Supervision for Scalable Discrete Image Generation](https://mlsanity.com/paper/2603.15150/) (2026)
- [The Benefits and Risks of Transductive Approaches for AI Fairness](https://mlsanity.com/paper/2406.12011/) (2024)
- [Neuromodulated Meta-Learning](https://mlsanity.com/paper/2411.06746/) (2024)
- [Learning from Ambiguous Data with Hard Labels](https://mlsanity.com/paper/2501.01844/) (2025)
- [Confidence-Based Annotation Of Brain Tumours In Ultrasound](https://mlsanity.com/paper/2502.15484/) (2025)
- [Combined Image Data Augmentations diminish the benefits of Adaptive Label Smoothing](https://mlsanity.com/paper/2507.16427/) (2025)

## Abstract (arXiv)

One-hot labels do not represent soft decision boundaries among concepts, and hence, models trained on them are prone to overfitting. Using soft labels as targets provide regularization, but different soft labels might be optimal at different stages of optimization. Also, training with fixed labels in the presence of noisy annotations leads to worse generalization. To address these limitations, we propose a framework, where we treat the labels as learnable parameters, and optimize them along with model parameters. The learned labels continuously adapt themselves to the model's state, thereby providing dynamic regularization. When applied to the task of supervised image-classification, our method leads to consistent gains across different datasets and architectures. For instance, dynamically learned labels improve ResNet18 by 2.1% on CIFAR100. When applied to dataset containing noisy labels, the learned labels correct the annotation mistakes, and improves over state-of-the-art by a significant margin. Finally, we show that learned labels capture semantic relationship between classes, and thereby improve teacher models for the downstream task of distillation.