mlsanityData to 14 Oct 2026

RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning

arXiv 2412.09858 ↗First posted 13 Dec 2024cs.RO cs.AI cs.LGcited by 47

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Recent advances in robotic foundation models have enabled the development of generalist policies that can adapt to diverse tasks. While these models show impressive flexibility, their performance heavily depends on the quality of their training data. In this work, we propose Reinforcement Learning Distilled Generalists (RLDG), a method that leverages reinforcement learning to generate high-quality training data for finetuning generalist policies. Through extensive real-world experiments on precise manipulation tasks like connector insertion and assembly, we demonstrate that generalist policies trained with RL-generated data consistently outperform those trained with human demonstrations, achieving up to 40% higher success rates while generalizing better to new tasks. We also provide a detailed analysis that reveals this performance gain stems from both optimized action distributions and improved state coverage. Our results suggest that combining task-specific RL with generalist policy distillation offers a promising approach for developing more capable and efficient robotic manipulation systems that maintain the flexibility of foundation models while achieving the performance of specialized controllers. Videos and code can be found on our project website https://generalist-distillation.github.io

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 27 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

23 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning29 Oct
  2. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning29 Jan

Other references · 21

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2016Progressive Neural Networkscited by 1,361
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  6. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  7. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  8. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  9. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  10. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
  11. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  12. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
9 more
  1. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  2. 2024FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learningcited by 80
  3. 2023Imitation Bootstrapped Reinforcement Learningcited by 64
  4. 2017Divide-and-Conquer Reinforcement Learningcited by 48
  5. 2019Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assemblycited by 39
  6. 2021Offline Meta-Reinforcement Learning for Industrial Insertioncited by 39
  7. 2021Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Studycited by 30
  8. 2023RLIF: Interactive Imitation Learning as Reinforcement Learningcited by 30
  9. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulationcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 23; 2026: 24.

47 papers.

Sort:

By area:

  1. 2025π*0.6: a VLA That Learns From Experiencecited by 303
  2. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 176
  3. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 154
  4. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
  5. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  6. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 69
  7. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 67
  8. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 49
  9. 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 40
  10. 2025World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulationcited by 35
  11. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Mergingcited by 22
  12. 2025Refined Policy Distillation: From VLA Generalists to RL Expertscited by 21
35 more
  1. 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Modelscited by 21
  2. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Modelscited by 18
  3. 2026Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policiescited by 15
  4. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  5. 2025Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learningcited by 8
  6. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8
  7. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  8. 2026ALOE: Action-Level Off-Policy Evaluation for Vision-Language-Action Model Post-Trainingcited by 7
  9. 2025Beyond Human Demonstrations: Diffusion-Based Reinforcement Learning to Generate Data for VLA Trainingcited by 5
  10. 2026IG-RFT: An Interaction-Guided RL Framework for VLA Models in Long-Horizon Robotic Manipulationcited by 5
  11. 2025CrystalGym: A New Benchmark for Materials Discovery Using Reinforcement Learningcited by 4
  12. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  13. 2025Integrating Diffusion-based Multi-task Learning with Online Reinforcement Learning for Robust Quadruped Robot Controlcited by 3
  14. 2025Discover, Learn, and Reinforce: Scaling Vision-Language-Action Pretraining with Diverse RL-Generated Trajectoriescited by 3
  15. 2026Learning Action Manifold with Multi-view Latent Priors for Robotic Manipulationcited by 3
  16. 2025Arnold: A multi-task, multi-embodiment muscle transformer policycited by 2
  17. 2026Squint: Fast Visual Reinforcement Learning for Sim-to-Real Roboticscited by 2
  18. 2026InSight: Self-Guided Skill Acquisition via Steerable VLAscited by 2
  19. 2026ExToken: Structured Exploration for Efficient Vision-Language-Action Reinforcement Fine-tuningcited by 2
  20. 2026Keyframe-Guided Structured Rewards for Reinforcement Learning in Long-Horizon Laboratory Roboticscited by 1
  21. 2026Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agentscited by 1
  22. 2026Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Modelscited by 1
  23. 2025EasyInsert: A Data-Efficient and Generalizable Insertion Policycited by 0
  24. 2026RoHIL: Robust Human-in-the-Loop Robotic Reinforcement Learning Against Illumination Variationscited by 0
  25. 2026Instrumentation for Imitation Learning: Enhancing Training Datasets for Clothes Hanger Insertioncited by 0
  26. 2026Multi-Objective Learning for Diffusion Models: A Statistical Theory under Semi-Supervised Learningcited by 0
  27. 2026VE2VF: Vision-Enabled to Vision-Free Distillation via Real-world Reinforcement Learning for Robust Contact-Rich Manipulationcited by 0
  28. 2026DexPIE: Stable Dexterous Policy Improvement from Real-World Experiencecited by 0
  29. 2026Scalable Multi-Task Data Generation via Reinforcement Learning for Language-Conditioned Bimanual Dexterous Manipulationcited by 0
  30. 2026SymVD: Symmetric Vision Language Action Distillation for Robot Manipulationcited by 0
  31. 2026DistAL: Distance-based Advantage Learning for VLA Fine-Tuningcited by 0
  32. 2026RouteRLT: Learning When and Which RL Specialist Should Control a Vision-Language-Action Policycited by 0
  33. 2026BEE: Intervention-Adaptive Real-World Reinforcement Learning with Vision-Language-Action Modelscited by 0
  34. 2026Dissecting Advantage-Guided Post-Training for Vision-Language-Action Policiescited by 0
  35. 2026Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance
  2. 2026Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience
  3. 2026Improving Robotic Generalist Policies via Flow Reversal Steering
  4. 2026Enhancing RL Generalizability in Robotics through SHAP Analysis of Algorithms and Hyperparameters
  5. 2025SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning
  6. 2022How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation
  7. 2024Effective Tuning Strategies for Generalist Robot Manipulation Policies
  8. 2025RM-RL: Role-Model Reinforcement Learning for Precise Robot Manipulation