mlsanityData to 7 Oct 2026

From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning

arXiv 2603.10263 ↗First posted 10 Mar 2026cs.RO cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce Distribution Contractive Reinforcement Learning (DICE-RL), a framework that uses reinforcement learning (RL) as a "distribution contraction" operator to refine pretrained generative robot policies. DICE-RL turns a pretrained behavior prior into a high-performing "pro" policy by amplifying high-success behaviors from online feedback. We pretrain a diffusion- or flow-based policy for broad behavioral coverage, then finetune it with a stable, sample-efficient residual off-policy RL framework that combines selective behavior regularization with value-guided action selection. Extensive experiments and analyses show that DICE-RL reliably improves performance with strong stability and sample efficiency. It enables mastery of complex long-horizon manipulation skills directly from high-dimensional pixel inputs, both in simulation and on a real robot. Project website: https://zhanyisun.github.io/dice.rl.2026/.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 52 of its 68 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

52 of 68 references.

References · 52

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  6. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  7. 2016End to End Learning for Self-Driving Carscited by 1,000
  8. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  9. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  10. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  11. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  12. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
38 more
  1. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  2. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  3. 2024Diffusion Policy Policy Optimizationcited by 255
  4. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  5. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  6. 2025Flow Q-Learningcited by 162
  7. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 151
  8. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  9. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 123
  10. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  11. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  12. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 102
  13. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  14. 2025Reinforcement Learning with Action Chunkingcited by 97
  15. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  16. 2025Flow Matching Policy Gradientscited by 86
  17. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
  18. 2023Score Regularized Policy Optimization through Diffusion Behaviorcited by 67
  19. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 64
  20. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
  21. 2023Imitation Bootstrapped Reinforcement Learningcited by 54
  22. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 51
  23. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 49
  24. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 47
  25. 2021Contraction Theory for Nonlinear Stability Analysis and Learning-based Control: A Tutorial Overviewcited by 42
  26. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  27. 2017A survey of robot learning from demonstrations for Human-Robot Collaborationcited by 38
  28. 2025CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learningcited by 34
  29. 2025EXPO: Stable Reinforcement Learning with Expressive Policiescited by 30
  30. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 29
  31. 2023CCIL: Continuity-based Data Augmentation for Corrective Imitation Learningcited by 26
  32. 2025The Pitfalls of Imitation Learning when Actions are Continuouscited by 25
  33. 2025The Coverage Principle: How Pre-Training Enables Post-Trainingcited by 25
  34. 2025Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distributioncited by 24
  35. 2022Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidancecited by 20
  36. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 20
  37. 2025Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Controlcited by 15
  38. 2025Reinforcement Learning for Flow-Matching Policiescited by 14

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Adapting Generalist Robot Policies with Semantic Reinforcement Learningcited by 3
  2. 2026RedFlow: Redirect Failure into Action-Level Corrections for Flow-matching VLA Policycited by 3
  3. 2026SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulationcited by 2
  4. 2026Adaptation of Generalist Robot Policies with Minimal Datacited by 1
  5. 2026HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RLcited by 1
  6. 2026Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latencycited by 1
  7. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewardscited by 0
  8. 2026Improving Online Reinforcement Learning via Bidirectional Behavior Prior Distillationcited by 0