# Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models

arXiv: https://arxiv.org/abs/2501.18592 · HTML: https://mlsanity.com/paper/2501.18592/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 227 of its 315 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Builds on (most cited first, 30 of 227)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)

## Built on by (most cited first, 23 of 23)

- [Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges](https://mlsanity.com/paper/2505.04769/) (2025)
- [TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning](https://mlsanity.com/paper/2510.03519/) (2025)
- [Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation](https://mlsanity.com/paper/2505.16985/) (2025)
- [Adapting Vision-Language Models Without Labels: A Comprehensive Survey](https://mlsanity.com/paper/2508.05547/) (2025)
- [Parallel Corpora for Machine Translation in Low-resource Indic Languages: A Comprehensive Review](https://mlsanity.com/paper/2503.04797/) (2025)
- [To Trust Or Not To Trust Your Vision-Language Model's Prediction](https://mlsanity.com/paper/2505.23745/) (2025)
- [Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability](https://mlsanity.com/paper/2508.04017/) (2025)
- [Towards Multimodal Domain Generalization with Few Labels](https://mlsanity.com/paper/2602.22917/) (2026)
- [Adaptive Confidence Regularization for Multimodal Failure Detection](https://mlsanity.com/paper/2603.02200/) (2026)
- [Balancing Multimodal Domain Generalization via Gradient Modulation and Projection](https://mlsanity.com/paper/2603.14175/) (2026)
- [Multi-source Multimodal Progressive Domain Adaption for Audio-Visual Deception Detection](https://mlsanity.com/paper/2508.12842/) (2025)
- [Efficient Generalization via Multimodal Co-Training under Data Scarcity and Distribution Shift](https://mlsanity.com/paper/2510.07509/) (2025)
- [VESSA: Video-based objEct-centric Self-Supervised Adaptation for Visual Foundation Models](https://mlsanity.com/paper/2510.20994/) (2025)
- [Multimodal Learning for Arcing Detection in Pantograph-Catenary Systems](https://mlsanity.com/paper/2602.08792/) (2026)
- [Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study](https://mlsanity.com/paper/2605.06643/) (2026)
- [Hystar: Hypernetwork-driven Style-adaptive Retrieval via Dynamic SVD Modulation](https://mlsanity.com/paper/2605.10009/) (2026)
- [Towards UAV Detection in the Real World: A New Multispectral Dataset UAVNet-MS and a New Method](https://mlsanity.com/paper/2605.20963/) (2026)
- [RADAR: Relative Angular Divergence Across Representations](https://mlsanity.com/paper/2605.23028/) (2026)
- [Unsupervised Collaborative Domain Adaptation for Driving Scene Parsing](https://mlsanity.com/paper/2606.01818/) (2026)
- [Trustworthy synthetic data for campaign decision support: strategy simulation fidelity and the PolicySynth framework](https://mlsanity.com/paper/2607.11269/) (2026)
- [M2LG-DG: A Multi-modal Local-Global Domain Generalization Framework for Cross-site Major Depressive Disorder Classification](https://mlsanity.com/paper/2609.09186/) (2026)
- [Robust Fault Detection in Mechanical Multimodal Time Series via Self-Supervised Cross-Modal Reconstruction](https://mlsanity.com/paper/2609.16314/) (2026)
- [Fusion Anything: A Generalized Multimodal Foundation Model](https://mlsanity.com/paper/2609.22107/) (2026)

## Abstract (arXiv)

In real-world scenarios, achieving domain adaptation and generalization poses significant challenges, as models must adapt to or generalize across unknown target distributions. Extending these capabilities to unseen multimodal distributions, i.e., multimodal domain adaptation and generalization, is even more challenging due to the distinct characteristics of different modalities. Significant progress has been made over the years, with applications ranging from action recognition to semantic segmentation. Besides, the recent advent of large-scale pre-trained multimodal foundation models, such as CLIP, has inspired works leveraging these models to enhance adaptation and generalization performances or adapting them to downstream tasks. This survey provides the first comprehensive review of recent advances from traditional approaches to foundation models, covering: (1) Multimodal domain adaptation; (2) Multimodal test-time adaptation; (3) Multimodal domain generalization; (4) Domain adaptation and generalization with the help of multimodal foundation models; and (5) Adaptation of multimodal foundation models. For each topic, we formally define the problem and thoroughly review existing methods. Additionally, we analyze relevant datasets and applications, highlighting open challenges and potential future research directions. We maintain an active repository that contains up-to-date literature at https://github.com/donghao51/Awesome-Multimodal-Adaptation.