# ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models

arXiv: https://arxiv.org/abs/2505.16517 · HTML: https://mlsanity.com/paper/2505.16517/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 34 of its 69 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 34)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model](https://mlsanity.com/paper/2504.07615/) (2025)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [Vision-Language Foundation Models as Effective Robot Imitators](https://mlsanity.com/paper/2311.01378/) (2023)
- [Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs](https://mlsanity.com/paper/2410.18451/) (2024)
- [LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL](https://mlsanity.com/paper/2503.07536/) (2025)
- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning](https://mlsanity.com/paper/2503.15558/) (2025)
- [RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete](https://mlsanity.com/paper/2502.21257/) (2025)
- [RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation](https://mlsanity.com/paper/2406.04339/) (2024)
- [LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning](https://mlsanity.com/paper/2406.11815/) (2024)
- [InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model](https://mlsanity.com/paper/2501.12368/) (2025)
- [AppAgent v2: Advanced Agent for Flexible Mobile Interactions](https://mlsanity.com/paper/2408.11824/) (2024)
- [Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection](https://mlsanity.com/paper/2412.04455/) (2024)
- [Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks](https://mlsanity.com/paper/2503.21696/) (2025)
- [On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspective](https://mlsanity.com/paper/2502.14296/) (2025)
- [Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning](https://mlsanity.com/paper/2504.12680/) (2025)
- [Geolocation with Real Human Gameplay Data: A Large-Scale Dataset and Human-Like Reasoning Framework](https://mlsanity.com/paper/2502.13759/) (2025)
- [SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation](https://mlsanity.com/paper/2310.16838/) (2023)
- [A Self-Correcting Vision-Language-Action Model for Fast and Slow System Manipulation](https://mlsanity.com/paper/2405.17418/) (2024)

## Built on by (most cited first, 27 of 27)

- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics](https://mlsanity.com/paper/2506.04308/) (2025)
- [Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2508.13073/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [VLA-R1: Enhancing Reasoning in Vision-Language-Action Models](https://mlsanity.com/paper/2510.01623/) (2025)
- [TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics](https://mlsanity.com/paper/2510.07181/) (2025)
- [Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model](https://mlsanity.com/paper/2508.06206/) (2025)
- [Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods](https://mlsanity.com/paper/2511.15722/) (2025)
- [DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Models](https://mlsanity.com/paper/2511.15669/) (2025)
- [EvoVLA: Self-Evolving Vision-Language-Action Model](https://mlsanity.com/paper/2511.16166/) (2025)
- [Why Tree-Style Branching Matters for Thought Advantage Estimation in GRPO](https://mlsanity.com/paper/2509.24494/) (2025)
- [StereoAdapter: Adapting Stereo Depth Estimation to Underwater Scenes](https://mlsanity.com/paper/2509.16415/) (2025)
- [VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Pottery](https://mlsanity.com/paper/2509.17191/) (2025)
- [MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles](https://mlsanity.com/paper/2510.00483/) (2025)
- [AnyDepth: Depth Estimation Made Easy](https://mlsanity.com/paper/2601.02760/) (2026)
- [VaseVQA-3D: Benchmarking 3D VLMs on Ancient Greek Pottery](https://mlsanity.com/paper/2510.04479/) (2025)
- [Affordance Agent Harness: Verification-Gated Skill Orchestration](https://mlsanity.com/paper/2605.00663/) (2026)
- [ANNIE: Be Careful of Your Robots](https://mlsanity.com/paper/2509.03383/) (2025)
- [3D CoCa v2: Contrastive Learners with Test-Time Search for Generalizable Spatial Intelligence](https://mlsanity.com/paper/2601.06496/) (2026)
- [RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models](https://mlsanity.com/paper/2603.21341/) (2026)
- [OmniDexVLG: Learning Dexterous Grasp Generation from Vision Language Model-Guided Grasp Semantics, Taxonomy and Functional Affordance](https://mlsanity.com/paper/2512.03874/) (2025)
- [NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models](https://mlsanity.com/paper/2603.09542/) (2026)
- [ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies](https://mlsanity.com/paper/2606.09630/) (2026)
- [Obstruction reasoning for robotic grasping](https://mlsanity.com/paper/2511.23186/) (2025)
- [Rethinking Agentic Reinforcement Learning In Large Language Models](https://mlsanity.com/paper/2604.27859/) (2026)
- [MERGE: Guided Vision-Language Models for Multi-Actor Event Reasoning and Grounding in Human-Robot Interaction](https://mlsanity.com/paper/2603.18988/) (2026)
- [Dual-Pathway Geometry-Aware MLLM for Spatial Intelligence](https://mlsanity.com/paper/2605.25334/) (2026)

## Abstract (arXiv)

Large Vision-Language Models (LVLMs) have recently advanced robotic manipulation by leveraging vision for scene perception and language for instruction following. However, existing methods rely heavily on costly human-annotated training datasets, which limits their generalization and causes them to struggle in out-of-domain (OOD) scenarios, reducing real-world adaptability. To address these challenges, we propose ManipLVM-R1, a novel reinforcement learning framework that replaces traditional supervision with Reinforcement Learning using Verifiable Rewards (RLVR). By directly optimizing for task-aligned outcomes, our method enhances generalization and physical reasoning while removing the dependence on costly annotations. Specifically, we design two rule-based reward functions targeting key robotic manipulation subtasks: an Affordance Perception Reward to enhance localization of interaction regions, and a Trajectory Match Reward to ensure the physical plausibility of action paths. These rewards provide immediate feedback and impose spatial-logical constraints, encouraging the model to go beyond shallow pattern matching and instead learn deeper, more systematic reasoning about physical interactions.