# QT-DoG: Quantization-aware Training for Domain Generalization

arXiv: https://arxiv.org/abs/2410.06020 · HTML: https://mlsanity.com/paper/2410.06020/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 80 of its 108 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 30 of 80)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Merging Models with Fisher-Weighted Averaging](https://mlsanity.com/paper/2111.09832/) (2021)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)

## Built on by (most cited first, 13 of 13)

- [ORANSight-2.0: Foundational LLMs for O-RAN](https://mlsanity.com/paper/2503.05200/) (2025)
- [Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques](https://mlsanity.com/paper/2411.06084/) (2024)
- [Modality-Balanced Collaborative Distillation for Multi-Modal Domain Generalization](https://mlsanity.com/paper/2511.20258/) (2025)
- [High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator](https://mlsanity.com/paper/2510.10693/) (2025)
- [Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracy](https://mlsanity.com/paper/2509.21173/) (2025)
- [What Survives When You Compress a Recursive Reasoner for the Edge?](https://mlsanity.com/paper/2606.26488/) (2026)
- [Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective](https://mlsanity.com/paper/2509.00859/) (2025)
- [Zero-Shot Quantization via Weight-Space Arithmetic](https://mlsanity.com/paper/2604.03420/) (2026)
- [Frequency Composition for Compressed and Domain-Adaptive Neural Networks](https://mlsanity.com/paper/2505.20890/) (2025)
- [Q2: Quantization-Aware Gradient Balancing and Attention Alignment for Low-Bit Quantization](https://mlsanity.com/paper/2511.05898/) (2025)
- [Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling](https://mlsanity.com/paper/2606.04920/) (2026)
- [Neural Network Quantization by Learning Low-Loss Subspaces](https://mlsanity.com/paper/2606.25087/) (2026)
- [Learning Subset-Shared Invariances for Domain Generalization with Mixture-of-Experts](https://mlsanity.com/paper/2606.25665/) (2026)

## Abstract (arXiv)

A key challenge in Domain Generalization (DG) is preventing overfitting to source domains, which can be mitigated by finding flatter minima in the loss landscape. In this work, we propose Quantization-aware Training for Domain Generalization (QT-DoG) and demonstrate that weight quantization effectively leads to flatter minima in the loss landscape, thereby enhancing domain generalization. Unlike traditional quantization methods focused on model compression, QT-DoG exploits quantization as an implicit regularizer by inducing noise in model weights, guiding the optimization process toward flatter minima that are less sensitive to perturbations and overfitting. We provide both an analytical perspective and empirical evidence demonstrating that quantization inherently encourages flatter minima, leading to better generalization across domains. Moreover, with the benefit of reducing the model size through quantization, we demonstrate that an ensemble of multiple quantized models further yields superior accuracy than the state-of-the-art DG approaches with no computational or memory overheads. Code is released at: https://saqibjaved1.github.io/QT_DoG/.