# Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLM-Powered Assistance

arXiv: https://arxiv.org/abs/2504.02901 · HTML: https://mlsanity.com/paper/2504.02901/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 3 Apr 2025.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Normalized Loss Functions for Deep Learning with Noisy Labels](https://mlsanity.com/paper/2006.13554/) (2020)
- [Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels](https://mlsanity.com/paper/1905.05040/) (2019)
- [Learning with Symmetric Label Noise: The Importance of Being Unhinged](https://mlsanity.com/paper/1505.07634/) (2015)
- [Text Classification via Large Language Models](https://mlsanity.com/paper/2305.08377/) (2023)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization](https://mlsanity.com/paper/2102.02414/) (2021)
- [Learning from Rules Generalizing Labeled Exemplars](https://mlsanity.com/paper/2004.06025/) (2020)
- [Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection](https://mlsanity.com/paper/2310.20046/) (2023)
- [Label Noise Types and Their Effects on Deep Learning](https://mlsanity.com/paper/2003.10471/) (2020)
- [NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction](https://mlsanity.com/paper/1909.02177/) (2019)
- [Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance](https://mlsanity.com/paper/2311.01108/) (2023)
- [From Noisy Prediction to True Label: Noisy Prediction Calibration via Generative Model](https://mlsanity.com/paper/2205.00690/) (2022)
- [Is BERT Robust to Label Noise? A Study on Learning with Noisy Labels in Text Classification](https://mlsanity.com/paper/2204.09371/) (2022)
- [Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency](https://mlsanity.com/paper/2403.18152/) (2024)

## Built on by (most cited first, 11 of 11)

- [Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation](https://mlsanity.com/paper/2506.03857/) (2025)
- [Active Testing of Large Language Models via Approximate Neyman Allocation](https://mlsanity.com/paper/2605.10075/) (2026)
- [Revisiting Active Learning under (Human) Label Variation](https://mlsanity.com/paper/2507.02593/) (2025)
- [Towards Multimodal Active Learning: Efficient Learning with Limited Paired Data](https://mlsanity.com/paper/2510.03247/) (2025)
- [Next Generation Active Learning: Mixture of LLMs in the Loop](https://mlsanity.com/paper/2601.15773/) (2026)
- [Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoE](https://mlsanity.com/paper/2505.23868/) (2025)
- [Beyond Semantic Relevance: Counterfactual Risk Minimization for Robust Retrieval-Augmented Generation](https://mlsanity.com/paper/2605.01302/) (2026)
- [Weed Out, Then Harvest: Dual Low-Rank Adaptation is an Effective Noisy Label Detector for Noise-Robust Learning](https://mlsanity.com/paper/2510.10208/) (2025)
- [MANTRA: a Framework for Multi-stage Adaptive Noise TReAtment During Training](https://mlsanity.com/paper/2512.04319/) (2025)
- [Combating Noisy Labels through Fostering Self- and Neighbor-Consistency](https://mlsanity.com/paper/2601.12795/) (2026)
- [Cross-Source Reasoning-based Correction for Author Name Disambiguation](https://mlsanity.com/paper/2606.08617/) (2026)

## Abstract (arXiv)

Learning from noisy labels (LNL) is a challenge that arises in many real-world scenarios where collected training data can contain incorrect or corrupted labels. Most existing solutions identify noisy labels and adopt active learning to query human experts on them for denoising. In the era of large language models (LLMs), although we can reduce the human effort to improve these methods, their performances are still subject to accurately separating the clean and noisy samples from noisy data. In this paper, we propose an innovative collaborative learning framework NoiseAL based on active learning to combine LLMs and small models (SMs) for learning from noisy labels. During collaborative training, we first adopt two SMs to form a co-prediction network and propose a dynamic-enhanced threshold strategy to divide the noisy data into different subsets, then select the clean and noisy samples from these subsets to feed the active annotator LLMs to rectify noisy samples. Finally, we employ different optimization objectives to conquer subsets with different degrees of label noises. Extensive experiments on synthetic and real-world noise datasets further demonstrate the superiority of our framework over state-of-the-art baselines.