mlsanityData to 14 Oct 2026

Consistency Regularization for Domain Generalization with Logit Attribution Matching

arXiv 2305.07888 ↗First posted 13 May 2023cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Domain generalization (DG) is about training models that generalize well under domain shift. Previous research on DG has been conducted mostly in single-source or multi-source settings. In this paper, we consider a third, lesser-known setting where a training domain is endowed with a collection of pairs of examples that share the same semantic information. Such semantic sharing (SS) pairs can be created via data augmentation and then utilized for consistency regularization (CR). We present a theory showing CR is conducive to DG and propose a novel CR method called Logit Attribution Matching (LAM). We conduct experiments on five DG benchmarks and four pretrained models with SS pairs created by both generic and targeted data augmentation methods. LAM outperforms representative single/multi-source DG methods and various CR methods that leverage SS pairs. The code and data of this project are available at https://github.com/Gaohan123/LAM

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 57 of its 77 references that are in the map. First posted on arXiv 13 May 2023.

Builds on

57 of 77 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  8. 2016Densely Connected Convolutional Networkscited by 6,720
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  11. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  12. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
38 more
  1. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  2. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  3. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  5. 2019Invariant Risk Minimizationcited by 1,854
  6. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  7. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  8. 2020In Search of Lost Domain Generalizationcited by 1,054
  9. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  10. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  11. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  12. 2021Domain Generalization: A Surveycited by 764
  13. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  14. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  15. 2018Recognition in Terra Incognitacited by 658
  16. 2013Domain Generalization via Invariant Feature Representationcited by 623
  17. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
  18. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  19. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 379
  20. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 351
  21. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
  22. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 336
  23. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 267
  24. 2022Contrastive Test-Time Adaptationcited by 261
  25. 2014Learning with Pseudo-Ensemblescited by 241
  26. 2020Domain Generalization using Causal Matchingcited by 225
  27. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  28. 2020Representation Learning via Invariant Causal Mechanismscited by 199
  29. 2021Deep Stable Learning for Out-Of-Distribution Generalizationcited by 153
  30. 2022Finetune like you pretrain: Improved finetuning of zero-shot vision modelscited by 150
  31. 2018Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objectscited by 143
  32. 2022Causality Inspired Representation Learning for Domain Generalizationcited by 138
  33. 2019Consistency Regularization for Generative Adversarial Networkscited by 136
  34. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
  35. 2021Causality-inspired Single-source Domain Generalization for Medical Image Segmentationcited by 119
  36. 2021Model-Based Domain Generalizationcited by 104
  37. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  38. 2021OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalizationcited by 86

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2023A Causal Framework to Unify Common Domain Generalization Approachescited by 3
  2. 2024Dual Risk Minimization: Towards Next-Level Robustness in Fine-tuning Zero-Shot Modelscited by 2
  3. 2025CLEAR: Unlearning Spurious Style-Content Associations with Contrastive LEarning with Anti-contrastive Regularizationcited by 1
  4. 2026One Task, Many Interfaces: Simple Cross-Interface Consistency Training Improves LLM Agent Robustnesscited by 0