# Robust Reinforcement Learning via Genetic Curriculum

arXiv: https://arxiv.org/abs/2202.08393 · HTML: https://mlsanity.com/paper/2202.08393/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models](https://mlsanity.com/paper/1805.06605/) (2018)
- [Automatic Goal Generation for Reinforcement Learning Agents](https://mlsanity.com/paper/1705.06366/) (2017)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [Reverse Curriculum Generation for Reinforcement Learning](https://mlsanity.com/paper/1707.05300/) (2017)
- [Action Robust Reinforcement Learning and Applications in Continuous Control](https://mlsanity.com/paper/1901.09184/) (2019)
- [Self-Supervised Policy Adaptation during Deployment](https://mlsanity.com/paper/2007.04309/) (2020)
- [Learning to Execute](https://mlsanity.com/paper/1410.4615/) (2014)
- [Adversarial Attacks and Defences Competition](https://mlsanity.com/paper/1804.00097/) (2018)
- [Risk Averse Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1904.00511/) (2019)
- [Self-Paced Deep Reinforcement Learning](https://mlsanity.com/paper/2004.11812/) (2020)
- [Learning Curriculum Policies for Reinforcement Learning](https://mlsanity.com/paper/1812.00285/) (2018)
- [BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning](https://mlsanity.com/paper/1806.06161/) (2018)
- [Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priors](https://mlsanity.com/paper/2003.04663/) (2020)
- [Accuracy-based Curriculum Learning in Deep Reinforcement Learning](https://mlsanity.com/paper/1806.09614/) (2018)
- [Fingerprint Policy Optimisation for Robust Reinforcement Learning](https://mlsanity.com/paper/1805.10662/) (2018)

## Built on by (most cited first, 10 of 10)

- [Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers](https://mlsanity.com/paper/2305.05909/) (2023)
- [Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula](https://mlsanity.com/paper/2311.01642/) (2023)
- [Robust Multi-agent Communication via Multi-view Message Certification](https://mlsanity.com/paper/2305.13936/) (2023)
- [CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2304.03535/) (2023)
- [Improving Environment Robustness of Deep Reinforcement Learning Approaches for Autonomous Racing Using Bayesian Optimization-based Curriculum Learning](https://mlsanity.com/paper/2312.10557/) (2023)
- [Physics-Informed Eikonal Caging for Whole-Arm Manipulation Planning](https://mlsanity.com/paper/2606.22143/) (2026)
- [Curricula for Learning Robust Policies with Factored State Representations in Changing Environments](https://mlsanity.com/paper/2409.09169/) (2024)
- [Scenario Generation for Risk-Aware Reinforcement Learning with Probably Approximately Safe Guarantees](https://mlsanity.com/paper/2606.04812/) (2026)
- [A Model-Driven Approach for Developing Families of Reinforcement Learning Environments](https://mlsanity.com/paper/2606.20324/) (2026)
- [Curriculum Generation under Structured Parametric Environments for Robust Navigation Policies](https://mlsanity.com/paper/2608.08545/) (2026)

## Abstract (arXiv)

Achieving robust performance is crucial when applying deep reinforcement learning (RL) in safety critical systems. Some of the state of the art approaches try to address the problem with adversarial agents, but these agents often require expert supervision to fine tune and prevent the adversary from becoming too challenging to the trainee agent. While other approaches involve automatically adjusting environment setups during training, they have been limited to simple environments where low-dimensional encodings can be used. Inspired by these approaches, we propose genetic curriculum, an algorithm that automatically identifies scenarios in which the agent currently fails and generates an associated curriculum to help the agent learn to solve the scenarios and acquire more robust behaviors. As a non-parametric optimizer, our approach uses a raw, non-fixed encoding of scenarios, reducing the need for expert supervision and allowing our algorithm to adapt to the changing performance of the agent. Our empirical studies show improvement in robustness over the existing state of the art algorithms, providing training curricula that result in agents being 2 - 8x times less likely to fail without sacrificing cumulative reward. We include an ablation study and share insights on why our algorithm outperforms prior approaches.