# RedCore: Relative Advantage Aware Cross-modal Representation Learning for Missing Modalities with Imbalanced Missing Rates

arXiv: https://arxiv.org/abs/2312.10386 · HTML: https://mlsanity.com/paper/2312.10386/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 16 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [What Makes Training Multi-Modal Classification Networks Hard?](https://mlsanity.com/paper/1905.12681/) (2019)
- [Balanced Multimodal Learning via On-the-fly Gradient Modulation](https://mlsanity.com/paper/2203.15332/) (2022)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [Trusted Multi-View Classification with Dynamic Evidential Fusion](https://mlsanity.com/paper/2204.11423/) (2022)
- [Provably Faster Algorithms for Bilevel Optimization](https://mlsanity.com/paper/2106.04692/) (2021)
- [M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2303.05302/) (2023)
- [Towards Good Practices for Missing Modality Robust Action Recognition](https://mlsanity.com/paper/2211.13916/) (2022)
- [On the consistency of supervised learning with missing values](https://mlsanity.com/paper/1902.06931/) (2019)
- [Doubly Aligned Incomplete Multi-view Clustering](https://mlsanity.com/paper/1903.02785/) (2019)
- [MMANet: Margin-aware Distillation and Modality-aware Regularization for Incomplete Multimodal Learning](https://mlsanity.com/paper/2304.08028/) (2023)
- [Semi-supervised Deep Generative Modelling of Incomplete Multi-Modality Emotional Data](https://mlsanity.com/paper/1808.02096/) (2018)
- [Fairness without Imputation: A Decision Tree Approach for Fair Prediction with Missing Values](https://mlsanity.com/paper/2109.10431/) (2021)
- [Assessing Fairness in the Presence of Missing Data](https://mlsanity.com/paper/2112.04899/) (2021)
- [Simultaneous Missing Value Imputation and Structure Learning with Groups](https://mlsanity.com/paper/2110.08223/) (2021)
- [Semi-supervised Multi-modal Emotion Recognition with Cross-Modal Distribution Matching](https://mlsanity.com/paper/2009.02598/) (2020)
- [Bilevel Optimization for Machine Learning: Algorithm Design and Convergence Analysis](https://mlsanity.com/paper/2108.00330/) (2021)
- [Gaussian Processes for Missing Value Imputation](https://mlsanity.com/paper/2204.04648/) (2022)
- [Learning Intuitive Physics with Multimodal Generative Models](https://mlsanity.com/paper/2101.04454/) (2021)
- [On Asymptotic Linear Convergence of Projected Gradient Descent for Constrained Least Squares](https://mlsanity.com/paper/2112.11760/) (2021)

## Built on by (most cited first, 15 of 15)

- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [PASSION: Towards Effective Incomplete Multi-Modal Medical Image Segmentation with Imbalanced Missing Rates](https://mlsanity.com/paper/2407.14796/) (2024)
- [MCE: Towards a General Framework for Handling Missing Modalities under Imbalanced Missing Rates](https://mlsanity.com/paper/2510.10534/) (2025)
- [Diffmv: A Unified Diffusion Framework for Healthcare Predictions with Random Missing Views and View Laziness](https://mlsanity.com/paper/2505.11802/) (2025)
- [TriSPrompt: A Hierarchical Soft Prompt Model for Multimodal Rumor Detection with Incomplete Modalities](https://mlsanity.com/paper/2509.19352/) (2025)
- [Boomda: Balanced Multi-objective Optimization for Multimodal Domain Adaptation](https://mlsanity.com/paper/2511.08152/) (2025)
- [MissBench: Benchmarking Multimodal Affective Analysis under Imbalanced Missing Modalities](https://mlsanity.com/paper/2603.09874/) (2026)
- [BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates](https://mlsanity.com/paper/2603.19718/) (2026)
- [Theoretical Analysis of Measure Consistency Regularization for Partially Observed Data](https://mlsanity.com/paper/2602.01437/) (2026)
- [A Clinical Point Cloud Paradigm for In-Hospital Mortality Prediction from Multi-Level Incomplete Multimodal EHRs](https://mlsanity.com/paper/2604.04614/) (2026)
- [Bridging the Missing-Modality Gap: Improving Text-Only Calibration of Vision Language Models](https://mlsanity.com/paper/2605.12517/) (2026)
- [Context-driven Missing-Modality Learning for Robust Medical Diagnosis with Image-Tabular Data](https://mlsanity.com/paper/2605.25968/) (2026)
- [General Incomplete Multimodal Learning via Dynamic Quality Perception](https://mlsanity.com/paper/2607.06943/) (2026)
- [GAUGE: Granularity-Adaptive Counterfactual Gating of Evidence for Incomplete Multimodal Classification](https://mlsanity.com/paper/2608.05608/) (2026)
- [Do More Modalities Always Help? A Geometric Perspective on Missing-Modality Robustness](https://mlsanity.com/paper/2610.04792/) (2026)

## Abstract (arXiv)

Multimodal learning is susceptible to modality missing, which poses a major obstacle for its practical applications and, thus, invigorates increasing research interest. In this paper, we investigate two challenging problems: 1) when modality missing exists in the training data, how to exploit the incomplete samples while guaranteeing that they are properly supervised? 2) when the missing rates of different modalities vary, causing or exacerbating the imbalance among modalities, how to address the imbalance and ensure all modalities are well-trained? To tackle these two challenges, we first introduce the variational information bottleneck (VIB) method for the cross-modal representation learning of missing modalities, which capitalizes on the available modalities and the labels as supervision. Then, accounting for the imbalanced missing rates, we define relative advantage to quantify the advantage of each modality over others. Accordingly, a bi-level optimization problem is formulated to adaptively regulate the supervision of all modalities during training. As a whole, the proposed approach features Relative advantage aware Cross-modal representation learning (abbreviated as RedCore) for missing modalities with imbalanced missing rates. Extensive empirical results demonstrate that RedCore outperforms competing models in that it exhibits superior robustness against either large or imbalanced missing rates.