mlsanityData to 14 Oct 2026

ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning

arXiv 2505.07395 ↗First posted 12 May 2025cs.ROcited by 47

On the map: Vision-language-action models → Reinforcement learning and post-training

also in 2 more
Abstract (arXiv)

Vision-Language-Action (VLA) models have shown great potential in general robotic decision-making tasks via imitation learning. However, the variable quality of training data often constrains the performance of these models. On the other hand, offline Reinforcement Learning (RL) excels at learning robust policy models from mixed-quality data. In this paper, we introduce Reinforced robot GPT (ReinboT), a novel end-to-end VLA model that integrates the RL principle of maximizing cumulative reward. ReinboT achieves a deeper understanding of the data quality distribution by predicting dense returns that capture the nuances of manipulation tasks. The dense return prediction capability enables the robot to generate more robust decision-making actions, oriented towards maximizing future benefits. Extensive experiments show that ReinboT achieves state-of-the-art performance on the CALVIN mixed-quality dataset and exhibits superior few-shot learning and out-of-distribution generalization capabilities in real-world tasks.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 42 of its 55 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

42 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models11 Mar
  2. 2025Improving Vision-Language-Action Model with Online Reinforcement Learning28 Jan
  3. 2024GRAPE: Generalizing Robot Policy via Preference Alignment28 Nov
  4. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance17 Oct

Other references · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  10. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  11. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  12. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
26 more
  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  2. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  3. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  4. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  5. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  6. 2023Learning Interactive Real-World Simulatorscited by 458
  7. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  8. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
  9. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  10. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  11. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 269
  12. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 161
  13. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  14. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 131
  15. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  16. 2024Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inferencecited by 97
  17. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  18. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 79
  19. 2023QUAR-VLA: Vision-Language-Action Model for Quadruped Robotscited by 56
  20. 2024Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learningcited by 47
  21. 2024GR-MG: Leveraging Partially Annotated Data via Multi-Modal Goal-Conditioned Policycited by 39
  22. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  23. 2024VLMPC: Vision-Language Model Predictive Control for Robotic Manipulationcited by 33
  24. 2025GEVRM: Goal-Expressive Video Generation Model For Robust Visual Manipulationcited by 27
  25. 2024Reinformer: Max-Return Sequence Modeling for Offline RLcited by 25
  26. 2025FDPP: Fine-tune Diffusion Policy with Human Preferencecited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 26.

47 papers.

Sort:

By area:

  1. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 213
  2. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 151
  3. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 67
  4. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 49
  5. 2025Long-VLA: Unleashing Long-Horizon Capability of Vision Language Action Model for Robot Manipulationcited by 48
  6. 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 40
  7. 2025CO-RFT: Efficient Fine-Tuning of Vision-Language-Action Models through Chunked Offline Reinforcement Learningcited by 37
  8. 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Modelscited by 24
  9. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 21
  10. 2025RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Modelscited by 18
  11. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  12. 2025Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Modelscited by 15
35 more
  1. 2026On-the-Fly VLA Adaptation via Test-Time Reinforcement Learningcited by 14
  2. 2026World-Value-Action Model: Implicit Planning for Vision-Language-Action Systemscited by 14
  3. 2025EvoVLA: Self-Evolving Vision-Language-Action Modelcited by 11
  4. 2025DEAS: DEtached value learning with Action Sequence for Scalable Offline RLcited by 10
  5. 2026DeltaVLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variationcited by 10
  6. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  7. 2025Reflection-Based Task Adaptation for Self-Improving VLAcited by 8
  8. 2026LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoningcited by 8
  9. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  10. 2026CRL-VLA: Continual Vision-Language-Action Learningcited by 7
  11. 2025RESample: A Robust Data Augmentation Framework via Exploratory Sampling for Robotic Manipulationcited by 6
  12. 2026TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulationcited by 6
  13. 2026UniSteer: Unified Noise Steering for Efficient Human-Guided VLA Adaptationcited by 6
  14. 2026IG-RFT: An Interaction-Guided RL Framework for VLA Models in Long-Horizon Robotic Manipulationcited by 5
  15. 2025VLA^2: Empowering Vision-Language-Action Models with an Agentic Framework for Unseen Concept Manipulationcited by 4
  16. 2026Self-Correcting VLA: Online Action Refinement via Sparse World Imaginationcited by 4
  17. 2026Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Modelscited by 4
  18. 2025Discover, Learn, and Reinforce: Scaling Vision-Language-Action Pretraining with Diverse RL-Generated Trajectoriescited by 3
  19. 2026WorldSample: Closed-loop Real-robot RL with World Modellingcited by 3
  20. 2025Beyond Success: Refining Elegant Robot Manipulation from Mixed-Quality Data via Just-in-Time Interventioncited by 2
  21. 2026Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Modelscited by 2
  22. 2026DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulationcited by 2
  23. 2026Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learningcited by 2
  24. 2025MUVLA: Learning to Explore Object Navigation via Map Understandingcited by 1
  25. 2026Keyframe-Guided Structured Rewards for Reinforcement Learning in Long-Horizon Laboratory Roboticscited by 1
  26. 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPOcited by 1
  27. 2026Trust Your Instincts: Confidence-Driven Test-Time RL for Vision-Language-Action Modelscited by 1
  28. 2026MobileVLA-R1 2.0: RL-Enhanced Reasoning for Mobile Robot Controlcited by 1
  29. 2026Boosting Vision-Language-Action Finetuning with Feasible Action Neighborhood Priorcited by 0
  30. 2026LM-X: Explainable Vision--Language--Action Modeling via Progress, Event, and Uncertainty Predictioncited by 0
  31. 2026Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Modelscited by 0
  32. 2026Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policiescited by 0
  33. 2026FailPatch: Failure Residual Patching for Vision-Language-Action Modelscited by 0
  34. 2026StructRL: Online Structured Reinforcement Learning for Long-Horizon Vision-Language-Action Taskscited by 0
  35. 2026Evo-RL: Iterative Robot Reinforcement Learning from Selective Experiencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning
  2. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy
  3. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models
  4. 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models
  5. 2025Reinforcing Action Policies by Prophesying
  6. 2026VLaRL: Augmenting Vision-Language-Action Models with Simulation-Trained Latent-Conditioned Residual RL
  7. 2026Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agents
  8. 2026HiL-ResRL: A Model-Agnostic Finetuning Adapter via Human-in-the-loop Residual Reinforcement Learning