Robust Model Based Reinforcement Learning Using L1 Adaptive Control
Not on the map. Why?
Abstract (arXiv)
We introduce L1-MBRL, a control-theoretic augmentation scheme for Model-Based Reinforcement Learning (MBRL) algorithms. Unlike model-free approaches, MBRL algorithms learn a model of the transition function using data and use it to design a control input. Our approach generates a series of approximate control-affine models of the learned transition function according to the proposed switching law. Using the approximate model, control input produced by the underlying MBRL is perturbed by the L1 adaptive control, which is designed to enhance the robustness of the system against uncertainties. Importantly, this approach is agnostic to the choice of MBRL algorithm, enabling the use of the scheme with various MBRL algorithms. MBRL algorithms with L1 augmentation exhibit enhanced performance and sample efficiency across multiple MuJoCo environments, outperforming the original MBRL algorithms, both with and without system noise.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 56 references that are in the map. First posted on arXiv 21 Mar 2024.
Builds on
32 of 56 references.
References · 32
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2015Trust Region Policy Optimizationcited by 2,503
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
- 2020MOPO: Model-based Offline Policy Optimizationcited by 630
- 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
- 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
- 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
- 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
20 more
- 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
- 2019Benchmarking Model-Based Reinforcement Learningcited by 158
- 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 156
- 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
- 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 99
- 2015Gaussian Processes for Data-Efficient Learning in Robotics and Controlcited by 97
- 2019Exploring Model-based Planning with Policy Networkscited by 90
- 2018Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RLcited by 80
- 2018A Study of Reinforcement Learning for Neural Machine Translationcited by 74
- 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 63
- 2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 61
- 2017Video Captioning via Hierarchical Reinforcement Learningcited by 47
- 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
- 2020Planning with Learned Dynamics: Probabilistic Guarantees on Safety and Reachability via Lipschitz Constantscited by 25
- 2019Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systemscited by 19
- 2023Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Functioncited by 19
- 2020L1-Adaptive MPPI Architecture for Robust and Agile Control of Multirotorscited by 14
- 2020Safe Feedback Motion Planning: A Contraction Theory and L1-Adaptive Control Based Approachcited by 11
- 2019Probabilistic Safety Constraints for Learned High Relative Degree System Dynamicscited by 9
Built on by
Citing papers per year: 2024: 1.
1 paper.