# Fidelity-Weighted Learning

arXiv: https://arxiv.org/abs/1711.02799 · HTML: https://mlsanity.com/paper/1711.02799/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2017.

## Built on by (most cited first, 19 of 19)

- [CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning](https://mlsanity.com/paper/2202.05613/) (2022)
- [Weighted Distillation with Unlabeled Examples](https://mlsanity.com/paper/2210.06711/) (2022)
- [Hundreds Guide Millions: Adaptive Offline Reinforcement Learning with Expert Guidance](https://mlsanity.com/paper/2309.01448/) (2023)
- [Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling](https://mlsanity.com/paper/2102.03551/) (2021)
- [Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking](https://mlsanity.com/paper/2105.09816/) (2021)
- [SLaM: Student-Label Mixing for Distillation with Unlabeled Examples](https://mlsanity.com/paper/2302.03806/) (2023)
- [Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data](https://mlsanity.com/paper/2411.08028/) (2024)
- [Multifidelity data fusion in convolutional encoder/decoder networks](https://mlsanity.com/paper/2205.05187/) (2022)
- [A Physics-Guided Bi-Fidelity Fourier-Featured Operator Learning Framework for Predicting Time Evolution of Drag and Lift Coefficients](https://mlsanity.com/paper/2311.03639/) (2023)
- [MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding](https://mlsanity.com/paper/2101.01881/) (2021)
- [WeaSuL: Weakly Supervised Dialogue Policy Learning: Reward Estimation for Multi-turn Dialogue](https://mlsanity.com/paper/2108.01487/) (2021)
- [Knowledge Distillation with Noisy Labels for Natural Language Understanding](https://mlsanity.com/paper/2109.10147/) (2021)
- [Label Augmentation with Reinforced Labeling for Weak Supervision](https://mlsanity.com/paper/2204.06436/) (2022)
- [Multihop: Leveraging Complex Models to Learn Accurate Simple Models](https://mlsanity.com/paper/2109.06961/) (2021)
- [Student as an Inherent Denoiser of Noisy Teacher](https://mlsanity.com/paper/2312.10185/) (2023)
- [Overlap-aware meta-learning attention to enhance hypergraph neural networks for node classification](https://mlsanity.com/paper/2503.07961/) (2025)
- [Component-Based Fairness in Face Attribute Classification with Bayesian Network-informed Meta Learning](https://mlsanity.com/paper/2505.01699/) (2025)
- [Weakly Supervised Medical Entity Extraction and Linking for Chief Complaints](https://mlsanity.com/paper/2509.01899/) (2025)
- [From Skeletons to Pixels: Few-Shot Precise Event Spotting via Representation and Prediction Distillation](https://mlsanity.com/paper/2604.22839/) (2026)

## Abstract (arXiv)

Training deep neural networks requires many training samples, but in practice training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing. This creates a fundamental quality versus-quantity trade-off in the learning process. Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data? We argue that if the learner could somehow know and take the label-quality into account when learning the data representation, we could get the best of both worlds. To this end, we propose "fidelity-weighted learning" (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data. FWL modulates the parameter updates to a student network (trained on the task we care about) on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher (who has access to the high-quality labels). Both student and teacher are learned from the data. We evaluate FWL on two tasks in information retrieval and natural language processing where we outperform state-of-the-art alternative semi-supervised methods, indicating that our approach makes better use of strong and weak labels, and leads to better task-dependent data representations.