mlsanityData to 14 Oct 2026

FLaRe: Achieving Masterful and Adaptive Robot Policies with Large-Scale Reinforcement Learning Fine-Tuning

arXiv 2409.16578 ↗First posted 25 Sep 2024cs.RO cs.CV cs.LGcited by 54

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

In recent years, the Robotics field has initiated several efforts toward building generalist robot policies through large-scale multi-task Behavior Cloning. However, direct deployments of these policies have led to unsatisfactory performance, where the policy struggles with unseen states and tasks. How can we break through the performance plateau of these models and elevate their capabilities to new heights? In this paper, we propose FLaRe, a large-scale Reinforcement Learning fine-tuning framework that integrates robust pre-trained representations, large-scale training, and gradient stabilization techniques. Our method aligns pre-trained policies towards task completion, achieving state-of-the-art (SoTA) performance both on previously demonstrated and on entirely novel tasks and embodiments. Specifically, on a set of long-horizon mobile manipulation tasks, FLaRe achieves an average success rate of 79.5% in unseen environments, with absolute improvements of +23.6% in simulation and +30.7% on real robots over prior SoTA methods. By utilizing only sparse rewards, our approach can enable generalizing to new capabilities beyond the pretraining data with minimal human effort. Moreover, we demonstrate rapid adaptation to new embodiments and behaviors with less than a day of fine-tuning. Videos can be found on the project website at https://robot-flare.github.io/

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 59 of its 71 references that are in the map. First posted on arXiv 25 Sep 2024.

Builds on

59 of 71 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes7 Aug
  2. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation6 Mar

Other references · 57

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Segment Anythingcited by 8,234
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  10. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  12. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
38 more
  1. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  2. 2017A Survey on Multi-Task Learningcited by 840
  3. 2019Solving Rubik's Cube with a Robot Handcited by 734
  4. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  5. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  6. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  7. 2022Efficiently Scaling Transformer Inferencecited by 539
  8. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  9. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  10. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  11. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  12. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  13. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  14. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  15. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  16. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 265
  17. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 237
  18. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  19. 2023ViNT: A Foundation Model for Visual Navigationcited by 222
  20. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 216
  21. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 214
  22. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 193
  23. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  24. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  25. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  26. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 117
  27. 2022Jump-Start Reinforcement Learningcited by 112
  28. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 111
  29. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  30. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 93
  31. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  32. 2018Robot Learning in Homes: Improving Generalization and Reducing Dataset Biascited by 75
  33. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 65
  34. 2023Imitation Bootstrapped Reinforcement Learningcited by 64
  35. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 58
  36. 2020AllenAct: A Framework for Embodied AI Researchcited by 54
  37. 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 48
  38. 2021AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scalecited by 44

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 30; 2026: 20.

54 papers.

Sort:

By area:

  1. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 176
  2. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 108
  3. 2025SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learningcited by 72
  4. 2025CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learningcited by 37
  5. 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Roboticscited by 33
  6. 2025DiWA: Diffusion Policy Adaptation with World Modelscited by 31
  7. 2025VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Modelscited by 31
  8. 2025Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Mergingcited by 22
  9. 2025RFTF: Reinforcement Fine-tuning for Embodied Agents with Temporal Feedbackcited by 20
  10. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Modelscited by 18
  11. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  12. 2026Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learningcited by 17
38 more
  1. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 16
  2. 2024FlowNav: Combining Flow Matching and Depth Priors for Efficient Navigationcited by 12
  3. 2025EvoVLA: Self-Evolving Vision-Language-Action Modelcited by 11
  4. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Modelscited by 10
  5. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
  6. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  7. 2026Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Modelscited by 9
  8. 2025VLN-Zero: Rapid Exploration and Cache-Enabled Neurosymbolic Vision-Language Planning for Zero-Shot Transfer in Robot Navigationcited by 8
  9. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8
  10. 2025SLAC: Safe and Efficient Real-Robot Reinforcement Learning via Unsupervised Simulation Pre-Trainingcited by 7
  11. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Reviewcited by 6
  12. 2025ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problemscited by 6
  13. 2025Quantization-Free Autoregressive Action Transformercited by 5
  14. 2025Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulationcited by 5
  15. 2025Constrained Decoding for Safe Robot Navigation Foundation Modelscited by 5
  16. 2025RoboSSM: Scalable In-context Imitation Learning via State-Space Modelscited by 5
  17. 2025ComposableNav: Instruction-Following Navigation in Dynamic Environments via Composable Diffusioncited by 4
  18. 2025VLA Model Post-Training via Action-Chunked PPO and Self Behavior Cloningcited by 4
  19. 2025Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Reviewcited by 3
  20. 2025Dual-Actor Fine-Tuning of VLA Models: A Talk-and-Tweak Human-in-the-Loop Approachcited by 3
  21. 2025MetricNet: Recovering Metric Scale in Generative Navigation Policiescited by 3
  22. 2025EmbodiedCoder: Parameterized Embodied Mobile Manipulation via Modern Coding Modelcited by 3
  23. 2025Discover, Learn, and Reinforce: Scaling Vision-Language-Action Pretraining with Diverse RL-Generated Trajectoriescited by 3
  24. 2026Generalizable Dense Reward for Long-Horizon Robotic Taskscited by 3
  25. 2026TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuningcited by 3
  26. 2026Adapting Generalist Robot Policies with Semantic Reinforcement Learningcited by 3
  27. 2026BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planningcited by 2
  28. 2026FORCE: Efficient VLA Reinforcement Fine-Tuning via Value-Calibrated Warm-up and Self-Distillationcited by 2
  29. 2026Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latencycited by 2
  30. 2025Synthetic vs. Real Training Data for Visual Navigationcited by 1
  31. 2025Shedding Light on VLN Robustness: A Black-box Framework for Indoor Lighting-based Adversarial Attackcited by 1
  32. 2026What to Ignore, What to React: Visually Robust RL Fine-Tuning of VLA Modelscited by 1
  33. 2026Learning Process Rewards via Success Visitation Matching for Efficient RLcited by 1
  34. 2026CounterAlign: Counterfactual Supervision for Vision-Language-Action Modelscited by 1
  35. 2026Specification-Aware Distribution Shaping for Robotics Foundation Modelscited by 0
  36. 2026Boosting Vision-Language-Action Finetuning with Feasible Action Neighborhood Priorcited by 0
  37. 2026Learn Where Outcomes Diverge: Efficient VLA RL via Probabilistic Chunk Maskingcited by 0
  38. 2026Foresight: Iterative Reasoning About Clues that Matter for Navigationcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025FLARE: Robot Learning with Implicit World Modeling
  2. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning
  3. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention
  4. 2026EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models
  5. 2025Rapidly Adapting Policies to the Real World via Simulation-Guided Fine-Tuning
  6. 2024Active Fine-Tuning of Multi-Task Policies
  7. 2025SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning
  8. 2026Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning