# Robust Domain Generalization for Multi-modal Object Recognition

arXiv: https://arxiv.org/abs/2408.05831 · HTML: https://mlsanity.com/paper/2408.05831/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 11 Aug 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Domain Generalization Using a Mixture of Multiple Latent Domains](https://mlsanity.com/paper/1911.07661/) (2019)
- [Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness](https://mlsanity.com/paper/2010.08001/) (2020)
- [CLIPood: Generalizing CLIP to Out-of-Distributions](https://mlsanity.com/paper/2302.00864/) (2023)
- [Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2204.02548/) (2022)
- [PLGSLAM: Progressive Neural Scene Represenation with Local to Global Bundle Adjustment](https://mlsanity.com/paper/2312.09866/) (2023)
- [A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation](https://mlsanity.com/paper/2403.11310/) (2024)

## Built on by (most cited first, 18 of 18)

- [Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation](https://mlsanity.com/paper/2409.09324/) (2024)
- [AR Overlay: Training Image Pose Estimation on Curved Surface in a Synthetic Way](https://mlsanity.com/paper/2409.14577/) (2024)
- [CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Survey](https://mlsanity.com/paper/2504.14280/) (2025)
- [EITNet: An IoT-Enhanced Framework for Real-Time Basketball Action Recognition](https://mlsanity.com/paper/2410.09954/) (2024)
- [Predicting 30-Day Hospital Readmission in Medicare Patients: Insights from an LSTM Deep Learning Model](https://mlsanity.com/paper/2410.17545/) (2024)
- [Real-time Monitoring of Lower Limb Movement Resistance Based on Deep Learning](https://mlsanity.com/paper/2410.19769/) (2024)
- [LTPNet Integration of Deep Learning and Environmental Decision Support Systems for Renewable Energy Demand Forecasting](https://mlsanity.com/paper/2410.15286/) (2024)
- [In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminates](https://mlsanity.com/paper/2410.15612/) (2024)
- [A GEN AI Framework for Medical Note Generation](https://mlsanity.com/paper/2410.01841/) (2024)
- [TRIZ Method for Urban Building Energy Optimization: GWO-SARIMA-LSTM Forecasting model](https://mlsanity.com/paper/2410.15283/) (2024)
- [Adaptive Self-Supervised Learning Strategies for Dynamic On-Device LLM Personalization](https://mlsanity.com/paper/2409.16973/) (2024)
- [GTA-Net: An IoT-Integrated 3D Human Pose Estimation System for Real-Time Adolescent Sports Posture Correction](https://mlsanity.com/paper/2411.06725/) (2024)
- [Evaluating Pre-trained Convolutional Neural Networks and Foundation Models as Feature Extractors for Content-based Medical Image Retrieval](https://mlsanity.com/paper/2409.09430/) (2024)
- [Research on Key Technologies for Cross-Cloud Federated Training of Large Language Models](https://mlsanity.com/paper/2410.19130/) (2024)
- [Enhancing Exchange Rate Forecasting with Explainable Deep Learning Models](https://mlsanity.com/paper/2410.19241/) (2024)
- [Detecting and Classifying Defective Products in Images Using YOLO](https://mlsanity.com/paper/2412.16935/) (2024)
- [Dynamic Attention and Bi-directional Fusion for Safety Helmet Wearing Detection](https://mlsanity.com/paper/2411.19071/) (2024)
- [Label-Decoupled Style Augmentation for Domain Generalization in Multi-Label Remote Sensing Scene Classification](https://mlsanity.com/paper/2607.12704/) (2026)

## Abstract (arXiv)

In multi-label classification, machine learning encounters the challenge of domain generalization when handling tasks with distributions differing from the training data. Existing approaches primarily focus on vision object recognition and neglect the integration of natural language. Recent advancements in vision-language pre-training leverage supervision from extensive visual-language pairs, enabling learning across diverse domains and enhancing recognition in multi-modal scenarios. However, these approaches face limitations in loss function utilization, generality across backbones, and class-aware visual fusion. This paper proposes solutions to these limitations by inferring the actual loss, broadening evaluations to larger vision-language backbones, and introducing Mixup-CLIPood, which incorporates a novel mix-up loss for enhanced class-aware visual fusion. Our method demonstrates superior performance in domain generalization across multiple datasets.