mlsanityData to 7 Oct 2026

PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assets

arXiv 2608.08053 ↗First posted 8 Aug 2026cs.RO cs.CVcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Simulation-ready 3D assets are central to robotics and embodied AI. Generating them from a single image is usually framed as a vision-language model that emits a serialized asset for a decoder to turn into geometry and physical fields, leaving the image-to-3D reasoning implicit. We argue the limiting factor is this output-centric view: part placement and local shape are entangled in one global-coordinate token stream, and the intermediate physical states are never exposed for supervision, conditioning, or verification. PhysX-CoT instead casts single-image asset generation as an explicit structured physical reasoning process, an ordered and machine-parseable trajectory of part-level states covering decomposition, 2D and 3D grounding, relations, coarse geometry, and surface cues that we separately supervise, use to condition geometry, and treat as reward targets. Geometry is factorized so that 3D boxes carry placement and local codes carry shape, and CoT-aligned GRPO optimizes parse validity, grounding, geometry, placement, and physical consistency. Under a unified protocol that retrains all learned baselines on the same backbone, data, and frozen decoder, PhysX-CoT outperforms the closest full-task baseline across geometry, scale, and physical-attribute metrics. Oracle, token-matched, and state-order controls show the explicit states are functional rather than cosmetic, and in Unreal Engine~5 the generated assets parse, collide, and articulate at high validity.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 53 references that are in the map. First posted on arXiv 8 Aug 2026.

Builds on

51 of 53 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought21 May
  2. 2026Training and Benchmarking Code Generation for Physics-Inspired Animations11 Feb
  3. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
  4. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 47

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2023Let's Verify Step by Stepcited by 3,598
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  9. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  10. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  12. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
35 more
  1. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  2. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  3. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  4. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  5. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  6. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  7. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  8. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  9. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  10. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  11. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  12. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  13. 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 140
  14. 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 106
  15. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 100
  16. 2024Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Modelcited by 90
  17. 2024LLaMA-Mesh: Unifying 3D Mesh Generation with Language Modelscited by 87
  18. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generationcited by 69
  19. 2024SINGAPO: Single Image Controlled Generation of Articulated Parts in Objectscited by 66
  20. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 60
  21. 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 40
  22. 2024Physically Compatible 3D Object Modeling from a Single Imagecited by 38
  23. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 34
  24. 2025EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligencecited by 32
  25. 2025PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Imagecited by 32
  26. 2025URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Modelcited by 20
  27. 2025Particulate: Feed-Forward 3D Object Articulationcited by 20
  28. 2024PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Imagecited by 19
  29. 2025Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assetscited by 19
  30. 2025MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Meshcited by 18
  31. 2026PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual Worldcited by 8
  32. 2026PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objectscited by 7
  33. 2026REST3D: Reconstructing Physically Stable 3D Scenes from a Single Imagecited by 6
  34. 2026SIMART: Decomposing Monolithic Meshes into Sim-ready Articulated Assets via MLLMcited by 3
  35. 2026AniGen: Unified S3 Fields for Animatable 3D Asset Generationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning
  2. 2026PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation
  3. 2026PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization
  4. 20263D Generation for Embodied AI and Robotic Simulation: A Survey
  5. 2025Scan, Materialize, Simulate: A Generalizable Framework for Physically Grounded Robot Planning
  6. 2026DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulation
  7. 2025PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assembly
  8. 2026Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Sim