# Smoothing the Shift: Towards Stable Test-Time Adaptation under Complex Multimodal Noises

arXiv: https://arxiv.org/abs/2503.02616 · HTML: https://mlsanity.com/paper/2503.02616/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Builds on (most cited first, 18 of 18)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [Robust Test-Time Adaptation in Dynamic Scenarios](https://mlsanity.com/paper/2303.13899/) (2023)
- [Contrastive Audio-Visual Masked Autoencoder](https://mlsanity.com/paper/2210.07839/) (2022)
- [Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factors](https://mlsanity.com/paper/2403.07366/) (2024)
- [MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.12667/) (2022)
- [SoTTA: Robust Test-Time Adaptation on Noisy Data Streams](https://mlsanity.com/paper/2310.10074/) (2023)
- [Patch-Mix Transformer for Unsupervised Domain Adaptation: A Game Perspective](https://mlsanity.com/paper/2303.13434/) (2023)
- [Classifier-guided Gradient Modulation for Enhanced Multimodal Learning](https://mlsanity.com/paper/2411.01409/) (2024)
- [Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition](https://mlsanity.com/paper/2407.05374/) (2024)
- [Towards Robust and Efficient Cloud-Edge Elastic Model Adaptation via Selective Entropy Distillation](https://mlsanity.com/paper/2402.17316/) (2024)
- [MedBN: Robust Test-Time Adaptation against Malicious Test Samples](https://mlsanity.com/paper/2403.19326/) (2024)
- [Low-rank Prompt Interaction for Continual Vision-Language Retrieval](https://mlsanity.com/paper/2501.14369/) (2025)

## Built on by (most cited first, 10 of 10)

- [textttAVROBUSTBENCH: Benchmarking the Robustness of Audio-Visual Recognition Models at Test-Time](https://mlsanity.com/paper/2506.00358/) (2025)
- [Bridging Modalities via Progressive Re-alignment for Multimodal Test-Time Adaptation](https://mlsanity.com/paper/2511.22862/) (2025)
- [Nip Rumors in the Bud: Retrieval-Guided Topic-Level Adaptation for Test-Time Fake News Video Detection](https://mlsanity.com/paper/2601.11981/) (2026)
- [Shedding the Facades, Connecting the Domains: Detecting Shifting Multimodal Hate Video with Test-Time Adaptation](https://mlsanity.com/paper/2602.00132/) (2026)
- [Audio-Visual Continual Test-Time Adaptation without Forgetting](https://mlsanity.com/paper/2602.18528/) (2026)
- [Decoupling Stability and Plasticity for Multi-Modal Test-Time Adaptation](https://mlsanity.com/paper/2603.00574/) (2026)
- [Hierarchical Adaptive networks with Task vectors for Test-Time Adaptation](https://mlsanity.com/paper/2508.09223/) (2025)
- [Test-Time Adaptation for Tactile-Vision-Language Models](https://mlsanity.com/paper/2602.15873/) (2026)
- [Multi-modal Test-time Adaptation via Adaptive Probabilistic Gaussian Calibration](https://mlsanity.com/paper/2604.19093/) (2026)
- [Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series](https://mlsanity.com/paper/2610.07499/) (2026)

## Abstract (arXiv)

Test-Time Adaptation (TTA) aims to tackle distribution shifts using unlabeled test data without access to the source data. In the context of multimodal data, there are more complex noise patterns than unimodal data such as simultaneous corruptions for multiple modalities and missing modalities. Besides, in real-world applications, corruptions from different distribution shifts are always mixed. Existing TTA methods always fail in such multimodal scenario because the abrupt distribution shifts will destroy the prior knowledge from the source model, thus leading to performance degradation. To this end, we reveal a new challenge named multimodal wild TTA. To address this challenging problem, we propose two novel strategies: sample identification with interquartile range Smoothing and unimodal assistance, and Mutual information sharing (SuMi). SuMi smooths the adaptation process by interquartile range which avoids the abrupt distribution shifts. Then, SuMi fully utilizes the unimodal features to select low-entropy samples with rich multimodal information for optimization. Furthermore, mutual information sharing is introduced to align the information, reduce the discrepancies and enhance the information utilization across different modalities. Extensive experiments on two public datasets show the effectiveness and superiority over existing methods under the complex noise patterns in multimodal data. Code is available at https://github.com/zrguo/SuMi.