mlsanityData to 7 Oct 2026

Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning

arXiv 2006.01096 ↗First posted 1 Jun 2020cs.LG cs.AI cs.ROcited by 37

Not on the map. Why?

Abstract (arXiv)

A fundamental challenge in reinforcement learning is to learn policies that generalize beyond the operating domains experienced during training. In this paper, we approach this challenge through the following invariance principle: an agent must find a representation such that there exists an action-predictor built on top of this representation that is simultaneously optimal across all training domains. Intuitively, the resulting invariant policy enhances generalization by finding causes of successful actions. We propose a novel learning algorithm, Invariant Policy Optimization (IPO), that implements this principle and learns an invariant policy during training. We compare our approach with standard policy gradient methods and demonstrate significant improvements in generalization performance on unseen domains for linear quadratic regulator and grid-world problems, and an example where a robot must learn to open doors with varying physical properties.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 5; 2023: 12; 2024: 5; 2025: 4; 2026: 1.

37 papers.

Sort:

By area:

  1. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  2. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  3. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  4. 2021Model-Based Domain Generalizationcited by 103
  5. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
  6. 2023Rethinking Domain Generalization for Face Anti-spoofing: Separability and Alignmentcited by 47
  7. 2023A Survey on Causal Reinforcement Learningcited by 43
  8. 2021Adversarial Robustness with Semi-Infinite Constrained Learningcited by 39
  9. 2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?cited by 36
  10. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  11. 2023Improving Generalization of Alignment with Human Preferences through Group Invariant Learningcited by 25
  12. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
25 more
  1. 2023Explore to Generalize in Zero-Shot RLcited by 20
  2. 2021When Is Generalizable Reinforcement Learning Tractable?cited by 19
  3. 2021Invariant Policy Learning: A Causal Perspectivecited by 17
  4. 2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
  5. 2023Invariant Causal Imitation Learning for Generalizable Policiescited by 15
  6. 2023Domain Generalization in Machine Learning Models for Wireless Communications: Concepts, State-of-the-Art, and Open Issuescited by 12
  7. 2025Evolution and The Knightian Blindspot of Machine Learningcited by 11
  8. 2023Generalization in Visual Reinforcement Learning with the Reward Sequence Distributioncited by 4
  9. 2022An Overview of Violence Detection Techniques: Current Challenges and Future Directionscited by 3
  10. 2025Learning General Policies with Policy Gradient Methodscited by 3
  11. 2024Learning Causally Invariant Reward Functions from Diverse Demonstrationscited by 2
  12. 2024Systems with Switching Causal Relations: A Meta-Causal Perspectivecited by 2
  13. 2021Learning Provably Robust Motion Planners Using Funnel Librariescited by 1
  14. 2023VIBR: Learning View-Invariant Value Functions for Robust Visual Controlcited by 1
  15. 2025Adaptive Data Exploitation in Deep Reinforcement Learningcited by 1
  16. 2025Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applicationscited by 1
  17. 2021Active Reinforcement Learning over MDPscited by 0
  18. 2022Improving generalization to new environments and removing catastrophic forgetting in Reinforcement Learning by using an eco-system of agentscited by 0
  19. 2022Language-Based Causal Representation Learningcited by 0
  20. 2022Improving generalization in reinforcement learning through forked agentscited by 0
  21. 2023Regularizing Adversarial Imitation Learning Using Causal Invariancecited by 0
  22. 2024Disentangled Representations for Causal Cognitioncited by 0
  23. 2024Invariant Correlation of Representation with Label: Enhancing Domain Generalization in Noisy Environmentscited by 0
  24. 2024Disentangling Recognition and Decision Regrets in Image-Based Reinforcement Learningcited by 0
  25. 2026Data Deletion Can Help in Adaptive RLcited by 0