# Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning

arXiv: https://arxiv.org/abs/2504.12680 · HTML: https://mlsanity.com/paper/2504.12680/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 27 of its 64 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 27 of 27)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking](https://mlsanity.com/paper/2501.04519/) (2025)
- [Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning](https://mlsanity.com/paper/2502.14768/) (2025)
- [MathPrompter: Mathematical Reasoning using Large Language Models](https://mlsanity.com/paper/2303.05398/) (2023)
- [Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI](https://mlsanity.com/paper/2407.06886/) (2024)
- [Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems](https://mlsanity.com/paper/2412.09413/) (2024)
- [Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition](https://mlsanity.com/paper/2501.03230/) (2024)
- [LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs](https://mlsanity.com/paper/2501.06186/) (2025)
- [A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness](https://mlsanity.com/paper/2411.03350/) (2024)
- [AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents](https://mlsanity.com/paper/2401.12963/) (2024)
- [Effective Prompt Extraction from Language Models](https://mlsanity.com/paper/2307.06865/) (2023)
- [Calibrating Large Language Models Using Their Generations Only](https://mlsanity.com/paper/2403.05973/) (2024)
- [EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment](https://mlsanity.com/paper/2410.09604/) (2024)
- [UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces](https://mlsanity.com/paper/2503.06157/) (2025)
- [VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI](https://mlsanity.com/paper/2410.11623/) (2024)
- [Purifying Large Language Models by Ensembling a Small Language Model](https://mlsanity.com/paper/2402.14845/) (2024)
- [video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model](https://mlsanity.com/paper/2502.11775/) (2025)
- [AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding](https://mlsanity.com/paper/2406.13807/) (2024)

## Built on by (most cited first, 30 of 37)

- [DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning](https://mlsanity.com/paper/2505.14362/) (2025)
- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [Grounded Reinforcement Learning for Visual Reasoning](https://mlsanity.com/paper/2505.23678/) (2025)
- [Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2508.13073/) (2025)
- [Look-Back: Implicit Visual Re-focusing in MLLM Reasoning](https://mlsanity.com/paper/2507.03019/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective](https://mlsanity.com/paper/2509.18905/) (2025)
- [Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks](https://mlsanity.com/paper/2510.25760/) (2025)
- [ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models](https://mlsanity.com/paper/2505.16517/) (2025)
- [Embodied AI: From LLMs to World Models](https://mlsanity.com/paper/2509.20021/) (2025)
- [Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods](https://mlsanity.com/paper/2511.15722/) (2025)
- [WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2506.07905/) (2025)
- [WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesis](https://mlsanity.com/paper/2507.04370/) (2025)
- [Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization](https://mlsanity.com/paper/2505.22038/) (2025)
- [DeepThinkVLA: Enhancing Reasoning Capability of Vision-Language-Action Models](https://mlsanity.com/paper/2511.15669/) (2025)
- [SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agents](https://mlsanity.com/paper/2506.21669/) (2025)
- [Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2505.18536/) (2025)
- [SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs](https://mlsanity.com/paper/2509.25390/) (2025)
- [Reinforced Reasoning for Embodied Planning](https://mlsanity.com/paper/2505.22050/) (2025)
- [Reinforcement Learning for Large Model: A Survey](https://mlsanity.com/paper/2508.08189/) (2025)
- [AirScape: An Aerial Generative World Model with Motion Controllability](https://mlsanity.com/paper/2507.08885/) (2025)
- [CoNav: Collaborative Cross-Modal Reasoning for Embodied Navigation](https://mlsanity.com/paper/2505.16663/) (2025)
- [EgoPrune: Efficient Token Pruning for Egomotion Video Reasoning in Embodied Agent](https://mlsanity.com/paper/2507.15428/) (2025)
- [Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoning](https://mlsanity.com/paper/2511.16160/) (2025)
- [ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replay](https://mlsanity.com/paper/2511.19033/) (2025)
- [ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Models](https://mlsanity.com/paper/2603.13033/) (2026)
- [Are Video Reasoning Models Ready to Go Outside?](https://mlsanity.com/paper/2603.10652/) (2026)
- [DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving](https://mlsanity.com/paper/2605.23176/) (2026)
- [Towards Accurate UAV Image Perception: Guiding Vision-Language Models with Stronger Task Prompts](https://mlsanity.com/paper/2512.07302/) (2025)
- [From Macro to Micro: Benchmarking Microscopic Spatial Intelligence on Molecules via Vision-Language Models](https://mlsanity.com/paper/2512.10867/) (2025)

## Abstract (arXiv)

Humans can perceive and reason about spatial relationships from sequential visual observations, such as egocentric video streams. However, how pretrained models acquire such abilities, especially high-level reasoning, remains unclear. This paper introduces Embodied-R, a collaborative framework combining large-scale Vision-Language Models (VLMs) for perception and small-scale Language Models (LMs) for reasoning. Using Reinforcement Learning (RL) with a novel reward system considering think-answer logical consistency, the model achieves slow-thinking capabilities with limited computational resources. After training on only 5k embodied video samples, Embodied-R with a 3B LM matches state-of-the-art multimodal reasoning models (OpenAI-o1, Gemini-2.5-pro) on both in-distribution and out-of-distribution embodied spatial reasoning tasks. Embodied-R also exhibits emergent thinking patterns such as systematic analysis and contextual integration. We further explore research questions including response length, training on VLM, strategies for reward design, and differences in model generalization after SFT (Supervised Fine-Tuning) and RL training.