# PAC-Bayesian Domain Adaptation Bounds for Multiclass Learners

arXiv: https://arxiv.org/abs/2207.05685 · HTML: https://mlsanity.com/paper/2207.05685/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 93 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data](https://mlsanity.com/paper/1703.11008/) (2017)
- [Fantastic Generalization Measures and Where to Find Them](https://mlsanity.com/paper/1912.02178/) (2019)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning](https://mlsanity.com/paper/1412.6614/) (2014)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Deep Domain-Adversarial Image Generation for Domain Generalisation](https://mlsanity.com/paper/2003.06054/) (2020)
- [Uniform convergence may be unable to explain generalization in deep learning](https://mlsanity.com/paper/1902.04742/) (2019)
- [Domain Generalization by Marginal Transfer Learning](https://mlsanity.com/paper/1711.07910/) (2017)
- [A Primer on PAC-Bayesian Learning](https://mlsanity.com/paper/1901.05353/) (2019)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach](https://mlsanity.com/paper/1804.05862/) (2018)
- [Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift](https://mlsanity.com/paper/2003.04475/) (2020)
- [Tighter risk certificates for neural networks](https://mlsanity.com/paper/2007.12911/) (2020)
- [A survey on domain adaptation theory: learning bounds and theoretical guarantees](https://mlsanity.com/paper/2004.11829/) (2020)
- [Theoretical Analysis of Domain Adaptation with Optimal Transport](https://mlsanity.com/paper/1610.04420/) (2016)
- [Implicit Regularization in Deep Learning](https://mlsanity.com/paper/1709.01953/) (2017)
- [Support and Invertibility in Domain-Invariant Representations](https://mlsanity.com/paper/1903.03448/) (2019)
- [In Search of Robust Measures of Generalization](https://mlsanity.com/paper/2010.11924/) (2020)
- [Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment](https://mlsanity.com/paper/1903.01689/) (2019)

## Built on by (most cited first, 13 of 13)

- [Few-shot Adaptation of Multi-modal Foundation Models: A Survey](https://mlsanity.com/paper/2401.01736/) (2024)
- [Your Data Is Not Perfect: Towards Cross-Domain Out-of-Distribution Detection in Class-Imbalanced Data](https://mlsanity.com/paper/2412.06284/) (2024)
- [Accounting for Sycophancy in Language Model Uncertainty Estimation](https://mlsanity.com/paper/2410.14746/) (2024)
- [LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue](https://mlsanity.com/paper/2210.07777/) (2022)
- [Out-Of-Domain Unlabeled Data Improves Generalization](https://mlsanity.com/paper/2310.00027/) (2023)
- [Bayesian Transfer Learning](https://mlsanity.com/paper/2312.13484/) (2023)
- [FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillation](https://mlsanity.com/paper/2405.17267/) (2024)
- [Fair Text Classification via Transferable Representations](https://mlsanity.com/paper/2503.07691/) (2025)
- [Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insights](https://mlsanity.com/paper/2207.07255/) (2022)
- [PAC-Bayesian Domain Adaptation Bounds for Multi-view learning](https://mlsanity.com/paper/2401.01048/) (2024)
- [Learning Transferable Topology Priors for Multi-Agent LLM Collaboration Across Domains](https://mlsanity.com/paper/2605.17359/) (2026)
- [ODD: Overlap-aware Estimation of Model Performance under Distribution Shift](https://mlsanity.com/paper/2506.14978/) (2025)
- [Probabilistic Geometric Alignment via Bayesian Latent Transport for Domain-Adaptive Foundation Models](https://mlsanity.com/paper/2603.23783/) (2026)

## Abstract (arXiv)

Multiclass neural networks are a common tool in modern unsupervised domain adaptation, yet an appropriate theoretical description for their non-uniform sample complexity is lacking in the adaptation literature. To fill this gap, we propose the first PAC-Bayesian adaptation bounds for multiclass learners. We facilitate practical use of our bounds by also proposing the first approximation techniques for the multiclass distribution divergences we consider. For divergences dependent on a Gibbs predictor, we propose additional PAC-Bayesian adaptation bounds which remove the need for inefficient Monte-Carlo estimation. Empirically, we test the efficacy of our proposed approximation techniques as well as some novel design-concepts which we include in our bounds. Finally, we apply our bounds to analyze a common adaptation algorithm that uses neural networks.