# Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration

arXiv: https://arxiv.org/abs/2605.03820 · HTML: https://mlsanity.com/paper/2605.03820/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 5 May 2026.

## Builds on (most cited first, 29 of 29)

- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Balanced Multimodal Learning via On-the-fly Gradient Modulation](https://mlsanity.com/paper/2203.15332/) (2022)
- [Trusted Multi-View Classification with Dynamic Evidential Fusion](https://mlsanity.com/paper/2204.11423/) (2022)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Provable Dynamic Fusion for Low-Quality Multimodal Data](https://mlsanity.com/paper/2306.02050/) (2023)
- [MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistance](https://mlsanity.com/paper/2405.17730/) (2024)
- [ReconBoost: Boosting Can Achieve Modality Reconcilement](https://mlsanity.com/paper/2405.09321/) (2024)
- [Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees](https://mlsanity.com/paper/2405.10301/) (2024)
- [Multimodal Fusion on Low-quality Data: A Comprehensive Survey](https://mlsanity.com/paper/2404.18947/) (2024)
- [Uncertainty Quantification of Collaborative Detection for Self-Driving](https://mlsanity.com/paper/2209.08162/) (2022)
- [Reliable Conflictive Multi-View Learning](https://mlsanity.com/paper/2402.16897/) (2024)
- [Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction](https://mlsanity.com/paper/2509.18658/) (2025)
- [Conformal Prediction with Learned Features](https://mlsanity.com/paper/2404.17487/) (2024)
- [Adaptive Unimodal Regulation for Balanced Multimodal Information Acquisition](https://mlsanity.com/paper/2503.18595/) (2025)
- [Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving](https://mlsanity.com/paper/2105.10266/) (2021)
- [Robust Multimodal Learning via Representation Decoupling](https://mlsanity.com/paper/2407.04458/) (2024)
- [Improving Multimodal Learning via Imbalanced Learning](https://mlsanity.com/paper/2507.10203/) (2025)
- [Boosting Multimodal Learning via Disentangled Gradient Learning](https://mlsanity.com/paper/2507.10213/) (2025)
- [Towards Balanced Active Learning for Multimodal Classification](https://mlsanity.com/paper/2306.08306/) (2023)
- [Mutual Information-calibrated Conformal Feature Fusion for Uncertainty-Aware Multimodal 3D Object Detection at the Edge](https://mlsanity.com/paper/2309.09593/) (2023)
- [Improving Multimodal Learning Balance and Sufficiency through Data Remixing](https://mlsanity.com/paper/2506.11550/) (2025)
- [Aligning Model Properties via Conformal Risk Control](https://mlsanity.com/paper/2406.18777/) (2024)
- [Conformalized Multimodal Uncertainty Regression and Reasoning](https://mlsanity.com/paper/2309.11018/) (2023)
- [Hyperdimensional Uncertainty Quantification for Multimodal Uncertainty Fusion in Autonomous Vehicles Perception](https://mlsanity.com/paper/2503.20011/) (2025)
- [Dynamic Evidence Decoupling for Trusted Multi-view Learning](https://mlsanity.com/paper/2410.03796/) (2024)
- [Any2Any: Incomplete Multimodal Retrieval with Conformal Prediction](https://mlsanity.com/paper/2411.10513/) (2024)
- [Multi-model Online Conformal Prediction with Graph-Structured Feedback](https://mlsanity.com/paper/2506.20898/) (2025)
- [Conformal Prediction for Multimodal Regression](https://mlsanity.com/paper/2410.19653/) (2024)

## Abstract (arXiv)

Multimodal learning often grapples with the challenge of low-quality data, which predominantly manifests as two facets: modality imbalance and noisy corruption. While these issues are often studied in isolation, we argue that they share a common root in the predictive uncertainty towards the reliability of individual modalities and instances during learning. In this paper, we propose a unified framework, termed Conformal Predictive Self-Calibration (CPSC), which leverages conformal prediction to equip the model with the ability to perform self-guided calibration on-the-fly. The core of our proposed CPSC lies in a novel self-calibrating training loop that seamlessly integrates two key modules: (1) Representation Self-Calibration, which decomposes unimodal features into components, and selectively fuses the most robust ones identified by a conformal predictor to enhance feature resilience. (2) Gradient Self-Calibration, which recalibrates the gradient flow during backpropagation based on instance-wise reliability scores, steering the optimization towards more trustworthy directions. Furthermore, we also devise a self-update strategy for the conformal predictor to ensure the entire system co-evolves consistently throughout the training process. Extensive experiments on six benchmark datasets under both imbalanced and noisy settings demonstrate that our CPSC framework consistently outperforms existing state-of-the-art methods. Our code is available at https://github.com/XunCHN/CPSC.