# Demonstration-Bootstrapped Autonomous Practicing via Multi-Task Reinforcement Learning

arXiv: https://arxiv.org/abs/2203.15755 · HTML: https://mlsanity.com/paper/2203.15755/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2022.

## Built on by (most cited first, 15 of 15)

- [From Play to Policy: Conditional Behavior Generation from Uncurated Robot Data](https://mlsanity.com/paper/2210.10047/) (2022)
- [Latent Plans for Task-Agnostic Offline Reinforcement Learning](https://mlsanity.com/paper/2209.08959/) (2022)
- [Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning](https://mlsanity.com/paper/2407.01531/) (2024)
- [Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning](https://mlsanity.com/paper/2310.15145/) (2023)
- [TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models](https://mlsanity.com/paper/2310.05905/) (2023)
- [Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning](https://mlsanity.com/paper/2303.01488/) (2023)
- [Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning](https://mlsanity.com/paper/2207.04703/) (2022)
- [You Only Live Once: Single-Life Reinforcement Learning](https://mlsanity.com/paper/2210.08863/) (2022)
- [Continuously Improving Mobile Manipulation with Autonomous Real-World RL](https://mlsanity.com/paper/2409.20568/) (2024)
- [Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Tasks](https://mlsanity.com/paper/2210.04476/) (2022)
- [Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback](https://mlsanity.com/paper/2310.20608/) (2023)
- [Intelligent Switching for Reset-Free RL](https://mlsanity.com/paper/2405.01684/) (2024)
- [When Learning Is Out of Reach, Reset: Generalization in Autonomous Visuomotor Reinforcement Learning](https://mlsanity.com/paper/2303.17600/) (2023)
- [From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models](https://mlsanity.com/paper/2606.00083/) (2026)
- [Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models](https://mlsanity.com/paper/2609.32069/) (2026)

## Abstract (arXiv)

Reinforcement learning systems have the potential to enable continuous improvement in unstructured environments, leveraging data collected autonomously. However, in practice these systems require significant amounts of instrumentation or human intervention to learn in the real world. In this work, we propose a system for reinforcement learning that leverages multi-task reinforcement learning bootstrapped with prior data to enable continuous autonomous practicing, minimizing the number of resets needed while being able to learn temporally extended behaviors. We show how appropriately provided prior data can help bootstrap both low-level multi-task policies and strategies for sequencing these tasks one after another to enable learning with minimal resets. This mechanism enables our robotic system to practice with minimal human intervention at training time while being able to solve long horizon tasks at test time. We show the efficacy of the proposed system on a challenging kitchen manipulation task both in simulation and in the real world, demonstrating the ability to practice autonomously in order to solve temporally extended problems.