Data to 7 Oct 2026

Jump-Start Reinforcement Learning

arXiv 2204.02372 ↗First posted 5 Apr 2022cs.LGcited by 102

Not on the map. Why?

Cited by 25 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning (RL) provides a theoretical framework for continuously improving an agent's behavior via trial and error. However, efficiently learning policies from scratch can be very difficult, particularly for tasks with exploration challenges. In such settings, it might be desirable to initialize RL with an existing policy, offline data, or demonstrations. However, naively performing such initialization in RL often works poorly, especially for value-based methods. In this paper, we present a meta algorithm that can use offline data, demonstrations, or a pre-existing policy to initialize an RL policy, and is compatible with any RL approach. In particular, we propose Jump-Start Reinforcement Learning (JSRL), an algorithm that employs two policies to solve tasks: a guide-policy, and an exploration-policy. By using the guide-policy to form a curriculum of starting states for the exploration-policy, we are able to efficiently improve performance on a set of simulated robotic tasks. We show via experiments that JSRL is able to significantly outperform existing imitation and reinforcement learning algorithms, particularly in the small-data regime. In addition, we provide an upper bound on the sample complexity of JSRL and show that with the help of a guide-policy, one can improve the sample complexity for non-optimism exploration methods from exponential in horizon to polynomial.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Builds on 40 of its 57 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

40 of 57 references.

References · 40

2013Playing Atari with Deep Reinforcement Learningcited by 3,211
2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
2018Is Q-learning Provably Efficient?cited by 485

28 more

2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 285
2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 275
2020First return, then explorecited by 256
2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
2022Online Decision Transformercited by 194
2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
2020Learning Near Optimal Policies with Low Inherent Bellman Errorcited by 180
2019Neural Policy Gradient Methods: Global Optimality and Rates of Convergencecited by 157
2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157
2020Almost Optimal Model-Free Reinforcement Learning via Reference-Advantage Decompositioncited by 127
2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 114
2020Variational Policy Gradient Method for Reinforcement Learning with General Utilitiescited by 108
2020PC-PG: Policy Cover Directed Exploration for Provable Policy Gradient Learningcited by 91
2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
2020Batch Policy Learning in Average Reward Markov Decision Processescited by 69
2017Learning Unknown Markov Decision Processes: A Thompson Sampling Approachcited by 65
2021AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scalecited by 43
2019Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adaptingcited by 42
2014Approximate Policy Iteration Schemes: A Comparisoncited by 39
2016Playing Atari Games with Deep Reinforcement Learning and Human Checkpoint Replaycited by 22
2018BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learningcited by 21
2019On Value Functions and the Agent-Environment Boundarycited by 19
2018Backplay: "Man muss immer umkehren"cited by 16
2018Solving the Rubik's Cube Without Human Knowledgecited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 15; 2024: 35; 2025: 22; 2026: 22.

102 papers.

Sort:

By area:

2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 158
2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 103
2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 91
2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
2025Self-Improving Vision-Language-Action Models with Data Generation via Residual RLcited by 72
2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 64
2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 58
2024FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuningcited by 54

38 more

2024Correlated Proxies: A New Definition and Improved Mitigation for Reward Hackingcited by 50
2023Dual RL: Unification and New Methods for Reinforcement and Imitation Learningcited by 48
2024Dataset Reset Policy Optimization for RLHFcited by 41
2024RIME: Robust Preference-based Reinforcement Learning with Noisy Preferencescited by 37
2023Inverse Reinforcement Learning without Reinforcement Learningcited by 35
2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 32
2023Large Language Model as a Policy Teacher for Training Reinforcement Learning Agentscited by 30
2023PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learningcited by 28
2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
2023Learning Realistic Traffic Agents in Closed-loopcited by 24
2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 23
2023Accelerating Exploration with Unlabeled Prior Datacited by 21
2022Phasic Self-Imitative Reduction for Sparse-Reward Goal-Conditioned Reinforcement Learningcited by 16
2026Reuse your FLOPs: Scaling RL on Hard Problems by Conditioning on Very Off-Policy Prefixescited by 16
2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstrationcited by 14
2024Leveraging Skills from Unlabeled Prior Data for Efficient Online Explorationcited by 13
2024Reverse Forward Curriculum Learning for Extreme Sample and Demonstration Efficiency in Reinforcement Learningcited by 12
2026PivotRL: High Accuracy Agentic Post-Training at Low Compute Costcited by 12
2023CQM: Curriculum Reinforcement Learning with a Quantized World Modelcited by 11
2024Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RLcited by 10
2023Residual Q-Learning: Offline and Online Policy Customization without Valuecited by 9
2024Optimistic Critic Reconstruction and Constrained Fine-Tuning for General Offline-to-Online RLcited by 9
2024Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformerscited by 8
2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8
2024RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanationcited by 7
2025Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Datacited by 7
2024DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delayscited by 6
2024Hybrid Reinforcement Learning from Offline Observation Alonecited by 5
2024Learning to Better Search with Language Models via Guided Reinforced Self-Trainingcited by 5
2025Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RLcited by 5
2025Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulationcited by 5
2024Enhancing Reinforcement Learning Agents with Local Guidescited by 4
2025Behavioral Exploration: Learning to Explore via In-Context Adaptationcited by 4
2025The Three Regimes of Offline-to-Online Reinforcement Learningcited by 4
2024Rocket Landing Control with Random Annealing Jump Start Reinforcement Learningcited by 3
2024Towards Fast Safe Online Reinforcement Learning via Policy Finetuningcited by 3
2025Adaptive Replay Buffer for Offline-to-Online Reinforcement Learningcited by 3
2026What Makes Value Learning Efficient in Residual Reinforcement Learning?cited by 3

50 most cited of 102 All 102 (JSON)