mlsanityData to 14 Oct 2026

Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning

arXiv 2503.15558 ↗First posted 18 Mar 2025cs.AI cs.CV cs.LGcited by 164

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Physical AI systems need to perceive, understand, and perform complex actions in the physical world. In this paper, we present the Cosmos-Reason1 models that can understand the physical world and generate appropriate embodied decisions (e.g., next step action) in natural language through long chain-of-thought reasoning processes. We begin by defining key capabilities for Physical AI reasoning, with a focus on physical common sense and embodied reasoning. To represent physical common sense, we use a hierarchical ontology that captures fundamental knowledge about space, time, and physics. For embodied reasoning, we rely on a two-dimensional ontology that generalizes across different physical embodiments. Building on these capabilities, we develop two multimodal large language models, Cosmos-Reason1-7B and Cosmos-Reason1-56B. We curate data and train our models in two stages: Physical AI supervised fine-tuning (SFT) and Physical AI reinforcement learning (RL). To evaluate our models, we build comprehensive benchmarks for physical common sense and embodied reasoning according to our ontologies. Evaluation results show that Physical AI SFT and RL bring significant improvements. To facilitate the development of Physical AI, we make our code and pre-trained models available under the NVIDIA Open Model License at https://github.com/nvidia-cosmos/cosmos-reason1.

Key facts. Cited by 164 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 64 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

51 of 64 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul
  2. 2023RoboVQA: Multimodal Long-Horizon Reasoning for Robotics1 Nov

Other references · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  10. 2025Qwen2.5-VL Technical Reportcited by 5,587
  11. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
37 more
  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  7. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  8. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  9. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  10. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  11. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  12. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
  13. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  14. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  15. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  16. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  17. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  18. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  19. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  20. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  21. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 221
  22. 2024RT-H: Action Hierarchies Using Languagecited by 210
  23. 2024An Empirical Study of Mamba-based Language Modelscited by 186
  24. 2023REFLECT: Summarizing Robot Experiences for Failure Explanation and Correctioncited by 175
  25. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
  26. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 165
  27. 2024Nemotron-4 340B Technical Reportcited by 144
  28. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 139
  29. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 130
  30. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 125
  31. 2025R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learningcited by 100
  32. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 96
  33. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 77
  34. 2024AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modelingcited by 77
  35. 2018IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoningcited by 73
  36. 2023Semantic Anomaly Detection with Large Language Modelscited by 53
  37. 2023An Adaptive Placement and Parallelism Framework for Accelerating RLHF Trainingcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 64; 2026: 100.

164 papers.

Sort:

By area:

  1. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
  2. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  3. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  4. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 100
  5. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  6. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learningcited by 87
  7. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 83
  8. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 76
  9. 20253D and 4D World Modeling: A Surveycited by 67
  10. 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 67
  11. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 62
  12. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 51
38 more
  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  2. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 44
  3. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Modelcited by 44
  4. 2025NFT: Bridging Supervised Learning and Reinforcement Learning in Math Reasoningcited by 38
  5. 2025VRAG: Learning World Models for Interactive Video Generationcited by 38
  6. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Modelscited by 38
  7. 2025Don't Blind Your VLA: Aligning Visual Representations for OOD Generalizationcited by 35
  8. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 34
  9. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  10. 2025MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learningcited by 30
  11. 2025Seeing the Arrow of Time in Large Multimodal Modelscited by 30
  12. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 28
  13. 2025ManipLVM-R1: Reinforcement Learning for Reasoning in Embodied Manipulation with Large Vision-Language Modelscited by 27
  14. 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Braincited by 25
  15. 2026WorldCompass: Reinforcement Learning for Long-Horizon World Modelscited by 24
  16. 2025Robix: A Unified Model for Robot Interaction, Reasoning and Planningcited by 23
  17. 2025Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoningcited by 23
  18. 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLcited by 21
  19. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Drivingcited by 21
  20. 2025QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Modelscited by 20
  21. 2026Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulationcited by 19
  22. 2025Contrastive Representation Regularization for Vision-Language-Action Modelscited by 17
  23. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  24. 2025A Step Toward World Models: A Survey on Robotic Manipulationcited by 15
  25. 2025Thinking in 360deg: Humanoid Visual Search in the Wildcited by 15
  26. 2025ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interactioncited by 15
  27. 2026Self-Refining Video Samplingcited by 15
  28. 2025From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Modelscited by 14
  29. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  30. 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMscited by 14
  31. 2025Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbolscited by 14
  32. 2025Data-regularized Reinforcement Learning for Diffusion Models at Scalecited by 13
  33. 2026Wall-OSS-0.5 Technical Reportcited by 12
  34. 2025DIP-R1: Deep Inspection and Perception with RL Looking Through and Understanding Complex Scenescited by 11
  35. 2025SURPRISE3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenescited by 11
  36. 2025RynnEC: Bringing MLLMs into Embodied Worldcited by 11
  37. 2025Point-It-Out: Benchmarking Embodied Reasoning for Vision Language Models in Multi-Stage Visual Groundingcited by 10
  38. 2025Pelican-VL 1.0: A Foundation Brain Model for Embodied Intelligencecited by 10

50 most cited of 164 All 164 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A little less conversation, a little more action, please: Investigating the physical common-sense of LLMs in a 3D embodied environment
  2. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI
  3. 2026Towards general embodied intelligence: integrating large language models, knowledge bases, and reasoning capabilities to build the next generation of AI agents
  4. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
  5. 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning
  6. 2025IPR-1: Interactive Physical Reasoner
  7. 2026Integrating Simulation and Chain-of-thought Reasoning in Multimodal-Language Models For Physical Reasoning
  8. 2025Embodied AI: From LLMs to World Models