# Continual Reinforcement Learning deployed in Real-life using Policy Distillation and Sim2Real Transfer

arXiv: https://arxiv.org/abs/1906.04452 · HTML: https://mlsanity.com/paper/1906.04452/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2019.

## Built on by (most cited first, 26 of 26)

- [Flow as the Cross-Domain Manipulation Interface](https://mlsanity.com/paper/2407.15208/) (2024)
- [Continual Learning with Foundation Models: An Empirical Study of Latent Replay](https://mlsanity.com/paper/2205.00329/) (2022)
- [Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation](https://mlsanity.com/paper/2305.02231/) (2023)
- [Visual Reinforcement Learning with Self-Supervised 3D Representations](https://mlsanity.com/paper/2210.07241/) (2022)
- [Reinforcement Learning for Ridesharing: An Extended Survey](https://mlsanity.com/paper/2105.01099/) (2021)
- [Continuum: Simple Management of Complex Continual Learning Scenarios](https://mlsanity.com/paper/2102.06253/) (2021)
- [CRIL: Continual Robot Imitation Learning via Generative and Prediction Model](https://mlsanity.com/paper/2106.09422/) (2021)
- [A Survey of Continual Reinforcement Learning](https://mlsanity.com/paper/2506.21872/) (2025)
- [QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer](https://mlsanity.com/paper/2404.07988/) (2024)
- [Beyond Supervised Continual Learning: a Review](https://mlsanity.com/paper/2208.14307/) (2022)
- [Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomization](https://mlsanity.com/paper/2207.14561/) (2022)
- [Continual Domain Randomization](https://mlsanity.com/paper/2403.12193/) (2024)
- [Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning](https://mlsanity.com/paper/2105.14780/) (2021)
- [Self-Activating Neural Ensembles for Continual Reinforcement Learning](https://mlsanity.com/paper/2301.00141/) (2022)
- [A Platform-Agnostic Deep Reinforcement Learning Framework for Effective Sim2Real Transfer towards Autonomous Driving](https://mlsanity.com/paper/2304.08235/) (2023)
- [Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents](https://mlsanity.com/paper/2203.08454/) (2022)
- [On the Verge of Solving Rocket League using Deep Reinforcement Learning and Sim-to-sim Transfer](https://mlsanity.com/paper/2205.05061/) (2022)
- [Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer](https://mlsanity.com/paper/2305.11589/) (2023)
- [Robust Navigation with Cross-Modal Fusion and Knowledge Transfer](https://mlsanity.com/paper/2309.13266/) (2023)
- [Data-Driven Simulator for Mechanical Circulatory Support with Domain Adversarial Neural Process](https://mlsanity.com/paper/2405.18536/) (2024)
- [Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learning](https://mlsanity.com/paper/2505.18447/) (2025)
- [Offline RL With Resource Constrained Online Deployment](https://mlsanity.com/paper/2110.03165/) (2021)
- [Using Curiosity for an Even Representation of Tasks in Continual Offline Reinforcement Learning](https://mlsanity.com/paper/2312.03177/) (2023)
- [Whisker-Inspired Tactile Sensing: A Sim2Real Approach for Precise Underwater Contact Tracking](https://mlsanity.com/paper/2410.14005/) (2024)
- [Learning more with the same effort: how randomization improves the robustness of a robotic deep reinforcement learning agent](https://mlsanity.com/paper/2501.14443/) (2025)
- [CIDER: Continual Interactive Distillation for Embodied Reinforcement Learning](https://mlsanity.com/paper/2608.21899/) (2026)

## Abstract (arXiv)

We focus on the problem of teaching a robot to solve tasks presented sequentially, i.e., in a continual learning scenario. The robot should be able to solve all tasks it has encountered, without forgetting past tasks. We provide preliminary work on applying Reinforcement Learning to such setting, on 2D navigation tasks for a 3 wheel omni-directional robot. Our approach takes advantage of state representation learning and policy distillation. Policies are trained using learned features as input, rather than raw observations, allowing better sample efficiency. Policy distillation is used to combine multiple policies into a single one that solves all encountered tasks.