mlsanityData to 7 Oct 2026

Linear unit-tests for invariance discovery

arXiv 2102.10867 ↗First posted 22 Feb 2021cs.LG cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

There is an increasing interest in algorithms to learn invariant correlations across training environments. A big share of the current proposals find theoretical support in the causality literature but, how useful are they in practice? The purpose of this note is to propose six linear low-dimensional problems -- unit tests -- to evaluate different types of out-of-distribution generalization in a precise manner. Following initial experiments, none of the three recently proposed alternatives passes all tests. By providing the code to automatically replicate all the results in this manuscript (https://www.github.com/facebookresearch/InvarianceUnitTests), we hope that our unit tests become a standard steppingstone for researchers in out-of-distribution generalization.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 9 of its 16 references that are in the map. First posted on arXiv 22 Feb 2021.

Builds on

9 of 16 references.

References · 9

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  5. 2019Invariant Risk Minimizationcited by 1,832
  6. 2018Recognition in Terra Incognitacited by 655
  7. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  8. 2018Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objectscited by 143
  9. 2020Learning explanations that are hard to varycited by 132

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 10; 2023: 5; 2024: 2; 2025: 2.

27 papers.

Sort:

  1. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  2. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  3. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  4. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  5. 2022Provable Domain Generalization via Invariant-Feature Subspace Recoverycited by 32
  6. 2021Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environmentscited by 30
  7. 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 29
  8. 2022Explicit Tradeoffs between Adversarial and Natural Distributional Robustnesscited by 20
  9. 2022"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shiftscited by 18
  10. 2023VNE: An Effective Method for Improving Deep Representation by Manipulating Eigenvalue Distributioncited by 17
  11. 2021On Invariance Penalties for Risk Minimizationcited by 10
  12. 2024Towards Robust Out-of-Distribution Generalization Bounds via Sharpnesscited by 10
15 more
  1. 2021Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensemblescited by 4
  2. 2023Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shiftcited by 4
  3. 2022Conditional entropy minimization principle for learning domain invariant representation featurescited by 3
  4. 2022Decorr: Environment Partitioning for Invariant Learning and OOD Generalizationcited by 3
  5. 2023Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddingscited by 3
  6. 2021Identifying and Benchmarking Natural Out-of-Context Prediction Problemscited by 2
  7. 2022Trade-off between reconstruction loss and feature alignment for domain generalizationcited by 1
  8. 2023Invariant-Feature Subspace Recovery: A New Class of Provable Domain Generalization Algorithmscited by 1
  9. 2021A call for better unit testing for invariant risk minimisationcited by 0
  10. 2021Invariant Risk Minimisation for Cross-Organism Inference: Substituting Mouse Data for Human Data in Human Risk Factor Discoverycited by 0
  11. 2022Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalizationcited by 0
  12. 2023Spuriosity Rankings for Free: A Simple Framework for Last Layer Retraining Based on Object Detectioncited by 0
  13. 2024Towards a Better Evaluation of Out-of-Domain Generalizationcited by 0
  14. 2025Feature Matching Intervention: Leveraging Observational Data for Causal Representation Learningcited by 0
  15. 2025PISA: Prioritized Invariant Subgraph Aggregation for Out-of-Distribution Generalization on Graphscited by 0