Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptation
Not on the map. Why?
Abstract (arXiv)
While Reinforcement Learning can achieve impressive results for complex tasks, the learned policies are generally prone to fail in downstream tasks with even minor model mismatch or unexpected perturbations. Recent works have demonstrated that a policy population with diverse behavior characteristics can generalize to downstream environments with various discrepancies. However, such policies might result in catastrophic damage during the deployment in practical scenarios like real-world systems due to the unrestricted behaviors of trained policies. Furthermore, training diverse policies without regulation of the behavior can result in inadequate feasible policies for extrapolating to a wide range of test conditions with dynamics shifts. In this work, we aim to train diverse policies under the regularization of the behavior patterns. We motivate our paradigm by observing the inverse dynamics in the environment with partial state information and propose Diversity in Regulation (DiR) training diverse policies with regulated behaviors to discover desired patterns that benefit the generalization. Considerable empirical results on various variations of different environments indicate that our method attains improvements over other diversity-driven counterparts.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 50 of its 62 references that are in the map. First posted on arXiv 24 Sep 2022.
Builds on
50 of 62 references.
References · 50
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2015Trust Region Policy Optimizationcited by 2,580
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2016Deep Variational Information Bottleneckcited by 930
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
- 2019Solving Rubik's Cube with a Robot Handcited by 734
- 2018Learning Dexterous In-Hand Manipulationcited by 687
- 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
- 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
- 2019Learning agile and dynamic motor skills for legged robotscited by 504
38 more
- 2017Robust Adversarial Reinforcement Learningcited by 451
- 2015Illuminating search spaces by mapping elitescited by 425
- 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
- 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
- 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
- 2014Robots that can adapt like animalscited by 204
- 2019Active Domain Randomizationcited by 170
- 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 168
- 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
- 2017Imagination-Augmented Agents for Deep Reinforcement Learningcited by 140
- 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
- 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 134
- 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
- 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 98
- 2019BayesSim: adaptive domain randomization via probabilistic inference for robotics simulatorscited by 85
- 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
- 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 78
- 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 68
- 2018Diversity-Driven Exploration Strategy for Deep Reinforcement Learningcited by 65
- 2020Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotioncited by 59
- 2021Auto-Tuned Sim-to-Real Transfercited by 49
- 2020Data-efficient Domain Randomization with Bayesian Optimizationcited by 47
- 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
- 2018Policy Transfer with Strategy Optimizationcited by 43
- 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 42
- 2020Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learningcited by 35
- 2019Diversity-Inducing Policy Gradient: Using Maximum Mean Discrepancy to Find a Set of Diverse Policiescited by 34
- 2022Context is Everything: Implicit Identification for Dynamics Adaptationcited by 27
- 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
- 2020The Emergence of Individualitycited by 22
- 2019Learning Novel Policies For Taskscited by 20
- 2021Discovering Diverse Nearly Optimal Policies with Successor Featurescited by 20
- 2021Discovering Diverse Solutions in Deep Reinforcement Learning by Maximizing State-Action-Based Mutual Informationcited by 19
- 2021Reinforcement Learning with Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotioncited by 14
- 2021Motion Planning by Learning the Solution Manifold in Trajectory Optimizationcited by 13
- 2021Adaptable Agent Populations via a Generative Model of Policiescited by 12
- 2022SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulationcited by 10
Built on by
Citing papers per year: 2023: 1.
1 paper.