mlsanityData to 7 Oct 2026

Invariant Policy Learning: A Causal Perspective

arXiv 2106.00808 ↗First posted 1 Jun 2021cs.LG cs.AI stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

Contextual bandit and reinforcement learning algorithms have been successfully used in various interactive learning systems such as online advertising, recommender systems, and dynamic pricing. However, they have yet to be widely adopted in high-stakes application domains, such as healthcare. One reason may be that existing approaches assume that the underlying mechanisms are static in the sense that they do not change over different environments. In many real-world systems, however, the mechanisms are subject to shifts across environments which may invalidate the static environment assumption. In this paper, we take a step toward tackling the problem of environmental shifts considering the framework of offline contextual bandits. We view the environmental shift problem through the lens of causality and propose multi-environment contextual bandits that allow for changes in the underlying mechanisms. We adopt the concept of invariance from the causality literature and introduce the notion of policy invariance. We argue that policy invariance is only relevant if unobserved variables are present and show that, in that case, an optimal invariant policy is guaranteed to generalize across environments under suitable assumptions. Our results establish concrete connections among causality, invariance, and contextual bandits.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 32 of its 89 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

32 of 89 references.

References · 32

  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2013Domain Generalization via Invariant Feature Representationcited by 622
  4. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  5. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  6. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  7. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  8. 2016Foundations of Structural Causal Models with Cycles and Latent Variablescited by 134
  9. 2017Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributionscited by 125
  10. 2018Anchor regression: heterogeneous data meets causalitycited by 116
  11. 2020Invariant Causal Prediction for Block MDPscited by 104
  12. 2016Causal Bandits: Learning Good Interventions via Causal Inferencecited by 100
20 more
  1. 2020A causal framework for distribution generalizationcited by 68
  2. 2019Off-Policy Evaluation in Partially Observable Environmentscited by 65
  3. 2018A Contextual Bandit Bake-offcited by 64
  4. 2018Offline Multi-Action Policy Learning: Generalization and Optimizationcited by 64
  5. 2017Invariant Causal Prediction for Sequential Datacited by 63
  6. 2018Policy Evaluation and Optimization with Continuous Treatmentscited by 57
  7. 2020Confounding-Robust Policy Evaluation in Infinite-Horizon Reinforcement Learningcited by 56
  8. 2017Balanced Policy Evaluation and Learningcited by 51
  9. 2013Strong Completeness and Faithfulness in Bayesian Networkscited by 44
  10. 2020Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learningcited by 37
  11. 2019Stabilizing Variable Selection and Regressioncited by 31
  12. 2016Contextual Bandits with Latent Confounders: An NMF Approachcited by 26
  13. 2018Causal Bandits with Propagating Inferencecited by 23
  14. 2020Bandits with Partially Observable Confounded Datacited by 22
  15. 2021Regularizing towards Causal Invariance: Linear Models with Proxiescited by 20
  16. 2018Learning stable and predictive structures in kinetic systems: Benefits of a causal approachcited by 12
  17. 2020Distributional robustness of K-class estimators and the PULSEcited by 11
  18. 2021Statistical Testing under Distributional Shiftscited by 6
  19. 2018Optimization over Continuous and Multi-dimensional Decisions with Observational Datacited by 5
  20. 2015Invariant Models for Causal Transfer Learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 6; 2024: 0; 2025: 4; 2026: 5.

17 papers.

Sort:

By area:

  1. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  2. 2023A Survey on Causal Reinforcement Learningcited by 43
  3. 2023Causal Reinforcement Learning: A Surveycited by 22
  4. 2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
  5. 2023Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RLcited by 10
  6. 2023Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confoundingcited by 7
  7. 2023Tackling Non-Stationarity in Reinforcement Learning via Causal-Origin Representationcited by 4
  8. 2025Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergiescited by 2
  9. 2025Bayesian Hierarchical Invariant Predictioncited by 1
  10. 2026Stable Blanket with Hidden Variables and Cyclescited by 1
  11. 2023Unfair Utilities and First Steps Towards Improving Themcited by 0
  12. 2025Invariant Federated Learning for Edge Intelligence: Mitigating Heterogeneity and Asynchrony via Exit Strategy and Invariant Penaltycited by 0
5 more
  1. 2025Evaluating and Learning Robust Bandit Policies Under Uncertain Causal Mechanismscited by 0
  2. 2026Anti-causal domain generalization: Leveraging unlabeled datacited by 0
  3. 2026Invariance-Based Dynamic Regret Minimizationcited by 0
  4. 2026Prediction-Intervention Games and Invariant Setscited by 0
  5. 2026Are Good Generators Good Decision-Makers? Policy Learning for General Interventions via Retargeted Counterfactual Generationcited by 0