# Domain Generalization via Gradient Surgery

arXiv: https://arxiv.org/abs/2108.01621 · HTML: https://mlsanity.com/paper/2108.01621/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 3 Aug 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [Improve Unsupervised Domain Adaptation with Mixup Training](https://mlsanity.com/paper/2001.00677/) (2020)
- [Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models](https://mlsanity.com/paper/2010.05874/) (2020)
- [Learning explanations that are hard to vary](https://mlsanity.com/paper/2009.00329/) (2020)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [Best sources forward: domain generalization through source-specific nets](https://mlsanity.com/paper/1806.05810/) (2018)

## Built on by (most cited first, 30 of 60)

- [Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02934/) (2021)
- [Sharpness-Aware Gradient Matching for Domain Generalization](https://mlsanity.com/paper/2303.10353/) (2023)
- [ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization](https://mlsanity.com/paper/2201.06910/) (2022)
- [Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization](https://mlsanity.com/paper/2308.06879/) (2023)
- [Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation](https://mlsanity.com/paper/2311.13209/) (2023)
- [Generalizable Decision Boundaries: Dualistic Meta-Learning for Open Set Domain Generalization](https://mlsanity.com/paper/2308.09391/) (2023)
- [Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches](https://mlsanity.com/paper/2203.14843/) (2022)
- [Domain-General Crowd Counting in Unseen Scenarios](https://mlsanity.com/paper/2212.02573/) (2022)
- [Single Domain Generalization for Crowd Counting](https://mlsanity.com/paper/2403.09124/) (2024)
- [Understanding Hessian Alignment for Domain Generalization](https://mlsanity.com/paper/2308.11778/) (2023)
- [Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization](https://mlsanity.com/paper/2401.10272/) (2024)
- [AutoGPart: Intermediate Supervision Search for Generalizable 3D Part Segmentation](https://mlsanity.com/paper/2203.06558/) (2022)
- [Diverse Target and Contribution Scheduling for Domain Generalization](https://mlsanity.com/paper/2309.16460/) (2023)
- [Gradient Estimation for Unseen Domain Risk Minimization with Pre-Trained Models](https://mlsanity.com/paper/2302.01497/) (2023)
- [Gradient-Guided Annealing for Domain Generalization](https://mlsanity.com/paper/2502.20162/) (2025)
- [PGrad: Learning Principal Gradients For Domain Generalization](https://mlsanity.com/paper/2305.01134/) (2023)
- [Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation](https://mlsanity.com/paper/2307.14709/) (2023)
- [Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Models](https://mlsanity.com/paper/2307.15049/) (2023)
- [Decoupled Prototype Learning for Reliable Test-Time Adaptation](https://mlsanity.com/paper/2401.08703/) (2024)
- [Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off](https://mlsanity.com/paper/2402.14648/) (2024)
- [Federated Unsupervised Domain Generalization using Global and Local Alignment of Gradients](https://mlsanity.com/paper/2405.16304/) (2024)
- [FIXED: Frustratingly Easy Domain Generalization with Mixup](https://mlsanity.com/paper/2211.05228/) (2022)
- [What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?](https://mlsanity.com/paper/2406.09908/) (2024)
- [Single Domain Generalization for Few-Shot Counting via Universal Representation Matching](https://mlsanity.com/paper/2505.16778/) (2025)
- [Point-to-Region Loss for Semi-Supervised Point-Based Crowd Counting](https://mlsanity.com/paper/2505.21943/) (2025)
- [GSMorph: Gradient Surgery for cine-MRI Cardiac Deformable Registration](https://mlsanity.com/paper/2306.14687/) (2023)
- [Taste More, Taste Better: Diverse Data and Strong Model Boost Semi-Supervised Crowd Counting](https://mlsanity.com/paper/2503.17984/) (2025)
- [Semi-Supervised Domain Generalization with Evolving Intermediate Domain](https://mlsanity.com/paper/2111.10221/) (2021)
- [Personalizing Intervened Network for Long-tailed Sequential User Behavior Modeling](https://mlsanity.com/paper/2208.09130/) (2022)
- [CNN Feature Map Augmentation for Single-Source Domain Generalization](https://mlsanity.com/paper/2305.16746/) (2023)

## Abstract (arXiv)

In real-life applications, machine learning models often face scenarios where there is a change in data distribution between training and test domains. When the aim is to make predictions on distributions different from those seen at training, we incur in a domain generalization problem. Methods to address this issue learn a model using data from multiple source domains, and then apply this model to the unseen target domain. Our hypothesis is that when training with multiple domains, conflicting gradients within each mini-batch contain information specific to the individual domains which is irrelevant to the others, including the test domain. If left untouched, such disagreement may degrade generalization performance. In this work, we characterize the conflicting gradients emerging in domain shift scenarios and devise novel gradient agreement strategies based on gradient surgery to alleviate their effect. We validate our approach in image classification tasks with three multi-domain datasets, showing the value of the proposed agreement strategy in enhancing the generalization capability of deep learning models in domain shift scenarios.