Data to 7 Oct 2026

EPOpt: Learning Robust Neural Network Policies Using Model Ensembles

arXiv 1610.01283 ↗First posted 5 Oct 2016cs.LG cs.AI cs.ROcited by 136

Not on the map. Why?

Abstract (arXiv)

Sample complexity and safety are major challenges when learning policies with reinforcement learning for real-world tasks, especially when the policies are represented using rich function approximators like deep neural networks. Model-based methods where the real-world target domain is approximated using a simulated source domain provide an avenue to tackle the above challenges by augmenting real data with simulated data. However, discrepancies between the simulated source domain and the target domain pose a challenge for simulated training. We introduce the EPOpt algorithm, which uses an ensemble of simulated source domains and a form of adversarial training to learn policies that are robust and generalize to a broad range of possible target domains, including unmodeled effects. Further, the probability distribution over source domains in the ensemble can be adapted using data from target domain and approximate Bayesian methods, to progressively make it a better approximation. Thus, learning on a model ensemble, along with source domain adaptation, provides the benefit of both robustness and learning/adaptation.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 26; 2022: 31; 2023: 32; 2024: 12; 2025: 16; 2026: 17.

136 papers.

Sort:

By area:

2021A Survey of Uncertainty in Deep Neural Networkscited by 675
2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
2021Online Robust Reinforcement Learning with Model Uncertaintycited by 95
2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothingcited by 83
2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 78

38 more

2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
2022Policy Gradient Method For Robust Reinforcement Learningcited by 67
2022When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learningcited by 59
2021Robot Learning from Randomized Simulations: A Reviewcited by 56
2022Efficient Risk-Averse Reinforcement Learningcited by 41
2021Adaptive-Control-Oriented Meta-Learning for Nonlinear Systemscited by 38
2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
2023Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaRcited by 32
2022Distributionally Robust Offline Reinforcement Learning with Linear Function Approximationcited by 31
2021Unsupervised Visual Attention and Invariance for Reinforcement Learningcited by 30
2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
2023Learning to Modulate pre-trained Models in RLcited by 28
2022Context is Everything: Implicit Identification for Dynamics Adaptationcited by 26
2021Block Contextual MDPs for Continual Learningcited by 25
2023Learning and Adapting Agile Locomotion Skills by Transferring Experiencecited by 25
2021Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Criticcited by 23
2023Robust Average-Reward Markov Decision Processescited by 22
2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 21
2025Model Predictive Task Sampling for Efficient and Robust Adaptationcited by 20
2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
2021Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimizationcited by 18
2021Unsupervised Reinforcement Learning in Multiple Environmentscited by 17
2022Control-oriented meta-learningcited by 17
2023Towards more Practical Threat Models in Artificial Intelligence Securitycited by 17
2022DROPO: Sim-to-Real Transfer with Offline Domain Randomizationcited by 16
2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 16
2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
2023H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gapscited by 15
2023AdaptNet: Policy Adaptation for Physics-Based Character Controlcited by 15
2024Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Modelscited by 15
2021Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learningcited by 14
2021Neural Network Verification in Controlcited by 14
2022Robust Constrained Reinforcement Learningcited by 14
2021On Effective Scheduling of Model-based Reinforcement Learningcited by 13
2021Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Groundingcited by 12
2023Risk-Averse Model Uncertainty for Distributionally Robust Safe Reinforcement Learningcited by 12

50 most cited of 136 All 136 (JSON)