# Is the Information Bottleneck Robust Enough? Towards Label-Noise Resistant Information Bottleneck Learning

arXiv: https://arxiv.org/abs/2512.10573 · HTML: https://mlsanity.com/paper/2512.10573/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Builds on (most cited first, 20 of 20)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)
- [Graph Information Bottleneck](https://mlsanity.com/paper/2010.12811/) (2020)
- [Nonlinear Information Bottleneck](https://mlsanity.com/paper/1705.02436/) (2017)
- [ProMix: Combating Label Noise via Maximizing Clean Sample Utility](https://mlsanity.com/paper/2207.10276/) (2022)
- [How does Early Stopping Help Generalization against Label Noise?](https://mlsanity.com/paper/1911.08059/) (2019)
- [Robust Information Bottleneck for Task-Oriented Communication with Digital Modulation](https://mlsanity.com/paper/2209.10382/) (2022)
- [Learning on Graphs under Label Noise](https://mlsanity.com/paper/2306.08194/) (2023)
- [Adaptive Sample Selection for Robust Learning under Label Noise](https://mlsanity.com/paper/2106.15292/) (2021)
- [NoisyGL: A Comprehensive Benchmark for Graph Neural Networks under Label Noise](https://mlsanity.com/paper/2406.04299/) (2024)
- [Combating Bilateral Edge Noise for Robust Link Prediction](https://mlsanity.com/paper/2311.01196/) (2023)
- [Extracting robust and accurate features via a robust information bottleneck](https://mlsanity.com/paper/1910.06893/) (2019)
- [Structured IB: Improving Information Bottleneck with Structured Feature Learning](https://mlsanity.com/paper/2412.08222/) (2024)

## Built on by (most cited first, 3 of 3)

- [NGDBench: Towards Neural Graph Data Management](https://mlsanity.com/paper/2603.05529/) (2026)
- [IBCapsNet: Information Bottleneck Capsule Network for Noise-Robust Representation Learning](https://mlsanity.com/paper/2603.20682/) (2026)
- [ST-TGExplainer: Disentangling Stability and Transition Patterns for Temporal GNN Interpretability](https://mlsanity.com/paper/2605.19822/) (2026)

## Abstract (arXiv)

The Information Bottleneck (IB) principle facilitates effective representation learning by preserving label-relevant information while compressing irrelevant information. However, its strong reliance on accurate labels makes it inherently vulnerable to label noise, prevalent in real-world scenarios, resulting in significant performance degradation and overfitting. To address this issue, we propose LaT-IB, a novel Label-Noise ResistanT Information Bottleneck method which introduces a "Minimal-Sufficient-Clean" (MSC) criterion. Instantiated as a mutual information regularizer to retain task-relevant information while discarding noise, MSC addresses standard IB's vulnerability to noisy label supervision. To achieve this, LaT-IB employs a noise-aware latent disentanglement that decomposes the latent representation into components aligned with to the clean label space and the noise space. Theoretically, we first derive mutual information bounds for each component of our objective including prediction, compression, and disentanglement, and moreover prove that optimizing it encourages representations invariant to input noise and separates clean and noisy label information. Furthermore, we design a three-phase training framework: Warmup, Knowledge Injection and Robust Training, to progressively guide the model toward noise-resistant representations. Extensive experiments demonstrate that LaT-IB achieves superior robustness and efficiency under label noise, significantly enhancing robustness and applicability in real-world scenarios with label noise.