mlsanityData to 7 Oct 2026

Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments

arXiv 2106.09913 ↗First posted 18 Jun 2021cs.LG stat.MLcited by 30

Not on the map. Why?

Abstract (arXiv)

Domain generalization aims at performing well on unseen test environments with data from a limited number of training environments. Despite a proliferation of proposal algorithms for this task, assessing their performance both theoretically and empirically is still very challenging. Distributional matching algorithms such as (Conditional) Domain Adversarial Networks [Ganin et al., 2016, Long et al., 2018] are popular and enjoy empirical success, but they lack formal guarantees. Other approaches such as Invariant Risk Minimization (IRM) require a prohibitively large number of training environments -- linear in the dimension of the spurious feature space ds -- even on simple data models like the one proposed by [Rosenfeld et al., 2021]. Under a variant of this model, we show that both ERM and IRM cannot generalize with o(ds) environments. We then present an iterative feature matching algorithm that is guaranteed with high probability to yield a predictor that generalizes after seeing only O(log ds) environments. Our results provide the first theoretical justification for a family of distribution-matching algorithms widely used in practice under a concrete nontrivial data model.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 18 Jun 2021.

Builds on

34 of 60 references.

References · 34

  1. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  4. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  5. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  6. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  7. 2020In Search of Lost Domain Generalizationcited by 1,042
  8. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  9. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  10. 2018Recognition in Terra Incognitacited by 655
  11. 2013Domain Generalization via Invariant Feature Representationcited by 622
  12. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
22 more
  1. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  2. 2015Return of Frustratingly Easy Domain Adaptationcited by 389
  3. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  4. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  5. 2020The Risks of Invariant Risk Minimizationcited by 262
  6. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
  7. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 245
  8. 2020Domain Generalization using Causal Matchingcited by 224
  9. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  10. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
  11. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  12. 2020Domain Adaptation with Conditional Distribution Matching and Generalized Label Shiftcited by 124
  13. 2021Does Invariant Risk Minimization Capture Invariance?cited by 116
  14. 2019Generalizing to unseen domains via distribution matchingcited by 109
  15. 2021Model-Based Domain Generalizationcited by 103
  16. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  17. 2017Label Efficient Learning of Transferable Representations across Domains and Taskscited by 49
  18. 2021Quantifying and Improving Transferability in Domain Generalizationcited by 35
  19. 2021An Online Learning Approach to Interpolation and Extrapolation in Domain Generalizationcited by 30
  20. 2021Linear unit-tests for invariance discoverycited by 27
  21. 2020Enforcing Predictive Invariance across Structured Biomedical Domainscited by 23
  22. 2020Risk Variance Penalizationcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 7; 2024: 7; 2025: 3; 2026: 2.

30 papers.

Sort:

  1. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  2. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  3. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  4. 2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 43
  5. 2021Near-Optimal Linear Regression under Distribution Shiftcited by 40
  6. 2022Provable Domain Generalization via Invariant-Feature Subspace Recoverycited by 32
  7. 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 29
  8. 2023Out-of-Domain Robustness via Targeted Augmentationscited by 24
  9. 2022Fair Representation Learning through Implicit Path Alignmentcited by 22
  10. 2023(Almost) Provable Error Bounds Under Distribution Shift via Disagreement Discrepancycited by 16
  11. 2024Feature contamination: Neural networks learn uncorrelated features and fail to generalizecited by 11
  12. 2022Domain Generalization with Correlated Style Uncertaintycited by 10
18 more
  1. 2024Controllable Prompt Tuning For Balancing Group Distributional Robustnesscited by 10
  2. 2022Malign Overfitting: Interpolation Can Provably Preclude Invariancecited by 6
  3. 2022Conditional entropy minimization principle for learning domain invariant representation featurescited by 3
  4. 2025Causality-Inspired Robustness for Nonlinear Models via Representation Learningcited by 3
  5. 2024Bridging Domains with Approximately Shared Featurescited by 2
  6. 2025Why Domain Generalization Fail? A View of Necessity and Sufficiencycited by 2
  7. 2022PAC Generalization via Invariant Representationscited by 1
  8. 2022Trade-off between reconstruction loss and feature alignment for domain generalizationcited by 1
  9. 2023Context is Environmentcited by 1
  10. 2023Invariant-Feature Subspace Recovery: A New Class of Provable Domain Generalization Algorithmscited by 1
  11. 2024Enhancing Fairness through Reweighting: A Path to Attain the Sufficiency Rulecited by 1
  12. 2025Set Valued Predictions For Robust Domain Generalizationcited by 1
  13. 2023Domain Invariant Learning for Gaussian Processes and Bayesian Explorationcited by 0
  14. 2024Preserving Silent Features for Domain Generalizationcited by 0
  15. 2024Beyond Discrepancy: A Closer Look at the Theory of Distribution Shiftcited by 0
  16. 2024Invariant Correlation of Representation with Label: Enhancing Domain Generalization in Noisy Environmentscited by 0
  17. 2026Beyond identifiability: Learning causal representations with few environments and finite samplescited by 0
  18. 2026Unveiling Invariant and Transferable Latent Factors Across Heterogeneous Environments via ATLAScited by 0