# Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning

arXiv: https://arxiv.org/abs/2503.15558 · HTML: https://mlsanity.com/paper/2503.15558/

## Key facts

- Cited by 164 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 51 of its 64 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 30 of 51)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)

## Built on by (most cited first, 30 of 164)

- [World Simulation with Video Foundation Models for Physical AI](https://mlsanity.com/paper/2511.00062/) (2025)
- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond](https://mlsanity.com/paper/2503.21614/) (2025)
- [Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models](https://mlsanity.com/paper/2504.03624/) (2025)
- [A Survey on Vision-Language-Action Models: An Action Tokenization Perspective](https://mlsanity.com/paper/2507.01925/) (2025)
- [AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning](https://mlsanity.com/paper/2505.16400/) (2025)
- [Grounded Reinforcement Learning for Visual Reasoning](https://mlsanity.com/paper/2505.23678/) (2025)
- [DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control](https://mlsanity.com/paper/2603.10448/) (2026)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [WoW: Towards a World omniscient World model Through Embodied Interaction](https://mlsanity.com/paper/2509.22642/) (2025)
- [GigaWorld-0: World Models as Data Engine to Empower Embodied AI](https://mlsanity.com/paper/2511.19861/) (2025)
- [A Survey: Learning Embodied Intelligence from Physical Simulators and World Models](https://mlsanity.com/paper/2507.00917/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks](https://mlsanity.com/paper/2503.21696/) (2025)
- [GigaBrain-0: A World Model-Powered Vision-Language-Action Model](https://mlsanity.com/paper/2510.19430/) (2025)
- [NFT: Bridging Supervised Learning and Reinforcement Learning in Math Reasoning](https://mlsanity.com/paper/2505.18116/) (2025)
- [VRAG: Learning World Models for Interactive Video Generation](https://mlsanity.com/paper/2505.21996/) (2025)
- [VLA-R1: Enhancing Reasoning in Vision-Language-Action Models](https://mlsanity.com/paper/2510.01623/) (2025)
- [Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization](https://mlsanity.com/paper/2510.25616/) (2025)
- [PAI-Bench: A Comprehensive Benchmark For Physical AI](https://mlsanity.com/paper/2512.01989/) (2025)
- [Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future](https://mlsanity.com/paper/2512.16760/) (2025)
- [MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learning](https://mlsanity.com/paper/2505.24871/) (2025)
- [Seeing the Arrow of Time in Large Multimodal Models](https://mlsanity.com/paper/2506.03340/) (2025)
- [LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding](https://mlsanity.com/paper/2605.27365/) (2026)
- [ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Models](https://mlsanity.com/paper/2505.16517/) (2025)
- [Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain](https://mlsanity.com/paper/2510.17801/) (2025)
- [WorldCompass: Reinforcement Learning for Long-Horizon World Models](https://mlsanity.com/paper/2602.09022/) (2026)
- [Robix: A Unified Model for Robot Interaction, Reasoning and Planning](https://mlsanity.com/paper/2509.01106/) (2025)
- [Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoning](https://mlsanity.com/paper/2510.11027/) (2025)
- [SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL](https://mlsanity.com/paper/2512.04069/) (2025)

## Abstract (arXiv)

Physical AI systems need to perceive, understand, and perform complex actions in the physical world. In this paper, we present the Cosmos-Reason1 models that can understand the physical world and generate appropriate embodied decisions (e.g., next step action) in natural language through long chain-of-thought reasoning processes. We begin by defining key capabilities for Physical AI reasoning, with a focus on physical common sense and embodied reasoning. To represent physical common sense, we use a hierarchical ontology that captures fundamental knowledge about space, time, and physics. For embodied reasoning, we rely on a two-dimensional ontology that generalizes across different physical embodiments. Building on these capabilities, we develop two multimodal large language models, Cosmos-Reason1-7B and Cosmos-Reason1-56B. We curate data and train our models in two stages: Physical AI supervised fine-tuning (SFT) and Physical AI reinforcement learning (RL). To evaluate our models, we build comprehensive benchmarks for physical common sense and embodied reasoning according to our ontologies. Evaluation results show that Physical AI SFT and RL bring significant improvements. To facilitate the development of Physical AI, we make our code and pre-trained models available under the NVIDIA Open Model License at https://github.com/nvidia-cosmos/cosmos-reason1.