# Uncertainty-Resilient Multimodal Learning via Consistency-Guided Cross-Modal Transfer

arXiv: https://arxiv.org/abs/2511.15741 · HTML: https://mlsanity.com/paper/2511.15741/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 18 Nov 2025.

## Builds on (most cited first, 15 of 15)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances](https://mlsanity.com/paper/2203.06935/) (2022)
- [CrossKD: Cross-Head Knowledge Distillation for Object Detection](https://mlsanity.com/paper/2306.11369/) (2023)
- [EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Model](https://mlsanity.com/paper/2401.10278/) (2024)
- [The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation](https://mlsanity.com/paper/2206.06487/) (2022)
- [Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition](https://mlsanity.com/paper/2112.01849/) (2021)
- [Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval](https://mlsanity.com/paper/2309.17093/) (2023)
- [MASA-TCN: Multi-anchor Space-aware Temporal Convolutional Neural Networks for Continuous and Discrete EEG Emotion Recognition](https://mlsanity.com/paper/2308.16207/) (2023)
- [Cross-Task Inconsistency Based Active Learning (CTIAL) for Emotion Recognition](https://mlsanity.com/paper/2412.01171/) (2024)
- [Multimodal Brain-Computer Interfaces: AI-powered Decoding Methodologies](https://mlsanity.com/paper/2502.02830/) (2025)

## Abstract (arXiv)

Multimodal learning systems often face substantial uncertainty due to noisy data, low-quality labels, and heterogeneous modality characteristics. These issues become especially critical in human-computer interaction settings, where data quality, semantic reliability, and annotation consistency vary across users and recording conditions. This thesis tackles these challenges by exploring uncertainty-resilient multimodal learning through consistency-guided cross-modal transfer. The central idea is to use cross-modal semantic consistency as a basis for robust representation learning. By projecting heterogeneous modalities into a shared latent space, the proposed framework mitigates modality gaps and uncovers structural relations that support uncertainty estimation and stable feature learning. Building on this foundation, the thesis investigates strategies to enhance semantic robustness, improve data efficiency, and reduce the impact of noise and imperfect supervision without relying on large, high-quality annotations. Experiments on multimodal affect-recognition benchmarks demonstrate that consistency-guided cross-modal transfer significantly improves model stability, discriminative ability, and robustness to noisy or incomplete supervision. Latent space analyses further show that the framework captures reliable cross-modal structure even under challenging conditions. Overall, this thesis offers a unified perspective on resilient multimodal learning by integrating uncertainty modeling, semantic alignment, and data-efficient supervision, providing practical insights for developing reliable and adaptive brain-computer interface systems.