mlsanityData to 7 Oct 2026

PAC-Bayesian Domain Adaptation Bounds for Multiclass Learners

arXiv 2207.05685 ↗First posted 12 Jul 2022cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Multiclass neural networks are a common tool in modern unsupervised domain adaptation, yet an appropriate theoretical description for their non-uniform sample complexity is lacking in the adaptation literature. To fill this gap, we propose the first PAC-Bayesian adaptation bounds for multiclass learners. We facilitate practical use of our bounds by also proposing the first approximation techniques for the multiclass distribution divergences we consider. For divergences dependent on a Gibbs predictor, we propose additional PAC-Bayesian adaptation bounds which remove the need for inefficient Monte-Carlo estimation. Empirically, we test the efficacy of our proposed approximation techniques as well as some novel design-concepts which we include in our bounds. Finally, we apply our bounds to analyze a common adaptation algorithm that uses neural networks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 37 of its 93 references that are in the map. First posted on arXiv 12 Jul 2022.

Builds on

37 of 93 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  5. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  6. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  7. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  8. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  9. 2016Deep Transfer Learning with Joint Adaptation Networkscited by 609
  10. 2017Exploring Generalization in Deep Learningcited by 595
  11. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  12. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
25 more
  1. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 364
  2. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  3. 2019Bridging Theory and Algorithm for Domain Adaptationcited by 332
  4. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 240
  5. 2020Deep Domain-Adversarial Image Generation for Domain Generalisationcited by 231
  6. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  7. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  8. 2019A Primer on PAC-Bayesian Learningcited by 130
  9. 2017Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributionscited by 125
  10. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 125
  11. 2020Domain Adaptation with Conditional Distribution Matching and Generalized Label Shiftcited by 124
  12. 2020Tighter risk certificates for neural networkscited by 112
  13. 2020A survey on domain adaptation theory: learning bounds and theoretical guaranteescited by 109
  14. 2016Theoretical Analysis of Domain Adaptation with Optimal Transportcited by 105
  15. 2017Implicit Regularization in Deep Learningcited by 90
  16. 2019Support and Invertibility in Domain-Invariant Representationscited by 87
  17. 2020In Search of Robust Measures of Generalizationcited by 79
  18. 2019Domain Adaptation with Asymmetrically-Relaxed Distribution Alignmentcited by 61
  19. 2015Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithmcited by 50
  20. 2013A PAC-Bayesian Tutorial with A Dropout Boundcited by 43
  21. 2015A New PAC-Bayesian Perspective on Domain Adaptationcited by 29
  22. 2020Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representationscited by 26
  23. 2018Unsupervised Domain Adaptation Based on Source-guided Discrepancycited by 20
  24. 2017Cross-lingual RST Discourse Parsingcited by 19
  25. 2017PAC-Bayes and Domain Adaptationcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 5; 2025: 2; 2026: 2.

13 papers.

Sort:

By area:

  1. 2024Few-shot Adaptation of Multi-modal Foundation Models: A Surveycited by 28
  2. 2024Your Data Is Not Perfect: Towards Cross-Domain Out-of-Distribution Detection in Class-Imbalanced Datacited by 21
  3. 2024Accounting for Sycophancy in Language Model Uncertainty Estimationcited by 18
  4. 2022LEATHER: A Framework for Learning to Generate Human-like Text in Dialoguecited by 4
  5. 2023Out-Of-Domain Unlabeled Data Improves Generalizationcited by 3
  6. 2023Bayesian Transfer Learningcited by 3
  7. 2024FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillationcited by 3
  8. 2025Fair Text Classification via Transferable Representationscited by 3
  9. 2022Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insightscited by 2
  10. 2024PAC-Bayesian Domain Adaptation Bounds for Multi-view learningcited by 1
  11. 2025ODD: Overlap-aware Estimation of Model Performance under Distribution Shiftcited by 1
  12. 2026Learning Transferable Topology Priors for Multi-Agent LLM Collaboration Across Domainscited by 1
1 more
  1. 2026Probabilistic Geometric Alignment via Bayesian Latent Transport for Domain-Adaptive Foundation Modelscited by 0