PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assets
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Simulation-ready 3D assets are central to robotics and embodied AI. Generating them from a single image is usually framed as a vision-language model that emits a serialized asset for a decoder to turn into geometry and physical fields, leaving the image-to-3D reasoning implicit. We argue the limiting factor is this output-centric view: part placement and local shape are entangled in one global-coordinate token stream, and the intermediate physical states are never exposed for supervision, conditioning, or verification. PhysX-CoT instead casts single-image asset generation as an explicit structured physical reasoning process, an ordered and machine-parseable trajectory of part-level states covering decomposition, 2D and 3D grounding, relations, coarse geometry, and surface cues that we separately supervise, use to condition geometry, and treat as reward targets. Geometry is factorized so that 3D boxes carry placement and local codes carry shape, and CoT-aligned GRPO optimizes parse validity, grounding, geometry, placement, and physical consistency. Under a unified protocol that retrains all learned baselines on the same backbone, data, and frozen decoder, PhysX-CoT outperforms the closest full-task baseline across geometry, scale, and physical-attribute metrics. Oracle, token-matched, and state-order controls show the explicit states are functional rather than cosmetic, and in Unreal Engine~5 the generated assets parse, collide, and articulate at high validity.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 53 references that are in the map. First posted on arXiv 8 Aug 2026.
Builds on
51 of 53 references.
Closest ancestors · 4
Same direction, newest first
- 2026PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought21 May
- 2026Training and Benchmarking Code Generation for Physics-Inspired Animations11 Feb
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
Other references · 47
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2023Let's Verify Step by Stepcited by 3,598
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
35 more
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
- 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
- 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
- 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
- 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
- 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
- 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 140
- 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 106
- 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 100
- 2024Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Modelcited by 90
- 2024LLaMA-Mesh: Unifying 3D Mesh Generation with Language Modelscited by 87
- 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generationcited by 69
- 2024SINGAPO: Single Image Controlled Generation of Articulated Parts in Objectscited by 66
- 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 60
- 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 40
- 2024Physically Compatible 3D Object Modeling from a Single Imagecited by 38
- 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 34
- 2025EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligencecited by 32
- 2025PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Imagecited by 32
- 2025URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Modelcited by 20
- 2025Particulate: Feed-Forward 3D Object Articulationcited by 20
- 2024PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Imagecited by 19
- 2025Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assetscited by 19
- 2025MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Meshcited by 18
- 2026PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual Worldcited by 8
- 2026PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objectscited by 7
- 2026REST3D: Reconstructing Physically Stable 3D Scenes from a Single Imagecited by 6
- 2026SIMART: Decomposing Monolithic Meshes into Sim-ready Articulated Assets via MLLMcited by 3
- 2026AniGen: Unified S3 Fields for Animatable 3D Asset Generationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning
- 2026PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation
- 2026PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization
- 20263D Generation for Embodied AI and Robotic Simulation: A Survey
- 2025Scan, Materialize, Simulate: A Generalizable Framework for Physically Grounded Robot Planning
- 2026DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulation
- 2025PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assembly
- 2026Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Sim