# Robustifying Reinforcement Learning Policies with L1 Adaptive Control

arXiv: https://arxiv.org/abs/2106.02249 · HTML: https://mlsanity.com/paper/2106.02249/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Overfitting in adversarially robust deep learning](https://mlsanity.com/paper/2002.11569/) (2020)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning](https://mlsanity.com/paper/1803.11347/) (2018)
- [Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL](https://mlsanity.com/paper/1812.07671/) (2018)
- [Deep Drone Racing: From Simulation to Reality with Domain Randomization](https://mlsanity.com/paper/1905.09727/) (2019)
- [Autonomous Drone Racing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08624/) (2021)
- [Meta Reinforcement Learning with Latent Variable Gaussian Processes](https://mlsanity.com/paper/1803.07551/) (2018)
- [Deep Drone Racing: Learning Agile Flight in Dynamic Environments](https://mlsanity.com/paper/1806.08548/) (2018)
- [Context-Aware Safe Reinforcement Learning for Non-Stationary Environments](https://mlsanity.com/paper/2101.00531/) (2021)
- [Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes](https://mlsanity.com/paper/2006.11441/) (2020)
- [L1-Adaptive MPPI Architecture for Robust and Agile Control of Multirotors](https://mlsanity.com/paper/2004.00152/) (2020)
- [Safe Feedback Motion Planning: A Contraction Theory and L1-Adaptive Control Based Approach](https://mlsanity.com/paper/2004.01142/) (2020)
- [L1-GP: L1 Adaptive Control with Bayesian Learning](https://mlsanity.com/paper/2004.14594/) (2020)
- [Probabilistic Safety Constraints for Learned High Relative Degree System Dynamics](https://mlsanity.com/paper/1912.10116/) (2019)
- [Adaptive Robust Quadratic Programs using Control Lyapunov and Barrier Functions](https://mlsanity.com/paper/2010.04699/) (2020)
- [MRAC-RL: A Framework for On-Line Policy Adaptation Under Parametric Model Uncertainty](https://mlsanity.com/paper/2011.10562/) (2020)

## Abstract (arXiv)

A reinforcement learning (RL) policy trained in a nominal environment could fail in a new/perturbed environment due to the existence of dynamic variations. Existing robust methods try to obtain a fixed policy for all envisioned dynamic variation scenarios through robust or adversarial training. These methods could lead to conservative performance due to emphasis on the worst case, and often involve tedious modifications to the training environment. We propose an approach to robustifying a pre-trained non-robust RL policy with L1 adaptive control. Leveraging the capability of an L1 control law in the fast estimation of and active compensation for dynamic variations, our approach can significantly improve the robustness of an RL policy trained in a standard (i.e., non-robust) way, either in a simulator or in the real world. Numerical experiments are provided to validate the efficacy of the proposed approach.