# A Closer Look at Smoothness in Domain Adversarial Training

arXiv: https://arxiv.org/abs/2206.08213 · HTML: https://mlsanity.com/paper/2206.08213/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 66 references that are in the map.
- First posted on arXiv 16 Jun 2022.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data](https://mlsanity.com/paper/1703.11008/) (2017)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [An Investigation into Neural Net Optimization via Hessian Eigenvalue Density](https://mlsanity.com/paper/1901.10159/) (2019)
- [PyHessian: Neural Networks Through the Lens of the Hessian](https://mlsanity.com/paper/1912.07145/) (2019)
- [Strong-Weak Distribution Alignment for Adaptive Object Detection](https://mlsanity.com/paper/1812.04798/) (2018)
- [When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations](https://mlsanity.com/paper/2106.01548/) (2021)
- [Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation](https://mlsanity.com/paper/1803.11365/) (2018)
- [Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering](https://mlsanity.com/paper/2003.08607/) (2020)
- [Gradually Vanishing Bridge for Adversarial Domain Adaptation](https://mlsanity.com/paper/2003.13183/) (2020)

## Built on by (most cited first, 30 of 85)

- [MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation](https://mlsanity.com/paper/2212.01322/) (2022)
- [Prompt-based Distribution Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2312.09553/) (2023)
- [Free Lunch for Domain Adversarial Training: Environment Label Smoothing](https://mlsanity.com/paper/2302.00194/) (2023)
- [ImbSAM: A Closer Look at Sharpness-Aware Minimization in Class-Imbalanced Recognition](https://mlsanity.com/paper/2308.07815/) (2023)
- [Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data](https://mlsanity.com/paper/2212.13827/) (2022)
- [Flatness-Aware Minimization for Domain Generalization](https://mlsanity.com/paper/2307.11108/) (2023)
- [SPA: A Graph Spectral Alignment Perspective for Domain Adaptation](https://mlsanity.com/paper/2310.17594/) (2023)
- [Subsidiary Prototype Alignment for Universal Domain Adaptation](https://mlsanity.com/paper/2210.15909/) (2022)
- [Simplifying Source-Free Domain Adaptation for Object Detection: Effective Self-Training Strategies and Performance Insights](https://mlsanity.com/paper/2407.07586/) (2024)
- [Split to Merge: Unifying Separated Modalities for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2403.06946/) (2024)
- [Make the U in UDA Matter: Invariant Consistency Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2309.12742/) (2023)
- [Unsupervised Domain Adaption Harnessing Vision-Language Pre-training](https://mlsanity.com/paper/2408.02192/) (2024)
- [Order-preserving Consistency Regularization for Domain Adaptation and Generalization](https://mlsanity.com/paper/2309.13258/) (2023)
- [Domain-Guided Conditional Diffusion Model for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2309.14360/) (2023)
- [Transferring to Real-World Layouts: A Depth-aware Framework for Scene Adaptation](https://mlsanity.com/paper/2311.12682/) (2023)
- [Enhancing Domain Adaptation through Prompt Gradient Alignment](https://mlsanity.com/paper/2406.09353/) (2024)
- [Generalizable Person Re-identification via Balancing Alignment and Uniformity](https://mlsanity.com/paper/2411.11471/) (2024)
- [Holistic Transfer: Towards Non-Disruptive Fine-Tuning with Partial Target Data](https://mlsanity.com/paper/2311.01420/) (2023)
- [Unknown Domain Inconsistency Minimization for Domain Generalization](https://mlsanity.com/paper/2403.07329/) (2024)
- [Generalizable Facial Expression Recognition](https://mlsanity.com/paper/2408.10614/) (2024)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [PHATNet: A Physics-guided Haze Transfer Network for Domain-adaptive Real-world Image Dehazing](https://mlsanity.com/paper/2507.14826/) (2025)
- [CLIP the Divergence: Language-guided Unsupervised Domain Adaptation](https://mlsanity.com/paper/2407.01842/) (2024)
- [Resolving Domain Shift For Representations Of Speech In Non-Invasive Brain Recordings](https://mlsanity.com/paper/2410.19986/) (2024)
- [Learning Transferable Conceptual Prototypes for Interpretable Unsupervised Domain Adaptation](https://mlsanity.com/paper/2310.08071/) (2023)
- [Combining inherent knowledge of vision-language models with unsupervised domain adaptation through strong-weak guidance](https://mlsanity.com/paper/2312.04066/) (2023)
- [Rethinking Guidance Information to Utilize Unlabeled Samples:A Label Encoding Perspective](https://mlsanity.com/paper/2406.02862/) (2024)
- [Dual-Path Adversarial Lifting for Domain Shift Correction in Online Test-time Adaptation](https://mlsanity.com/paper/2408.13983/) (2024)
- [Supervision Interpolation via LossMix: Generalizing Mixup for Object Detection and Beyond](https://mlsanity.com/paper/2303.10343/) (2023)
- [Continual Domain Adaptation through Pruning-aided Domain-specific Weight Modulation](https://mlsanity.com/paper/2304.07560/) (2023)

## Abstract (arXiv)

Domain adversarial training has been ubiquitous for achieving invariant representations and is used widely for various domain adaptation tasks. In recent times, methods converging to smooth optima have shown improved generalization for supervised learning tasks like classification. In this work, we analyze the effect of smoothness enhancing formulations on domain adversarial training, the objective of which is a combination of task loss (eg. classification, regression, etc.) and adversarial terms. We find that converging to a smooth minima with respect to (w.r.t.) task loss stabilizes the adversarial training leading to better performance on target domain. In contrast to task loss, our analysis shows that converging to smooth minima w.r.t. adversarial loss leads to sub-optimal generalization on the target domain. Based on the analysis, we introduce the Smooth Domain Adversarial Training (SDAT) procedure, which effectively enhances the performance of existing domain adversarial methods for both classification and object detection tasks. Our analysis also provides insight into the extensive usage of SGD over Adam in the community for domain adversarial training.