mlsanityData to 14 Oct 2026

LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstruction

arXiv 2610.01863 ↗First posted 1 Oct 2026cs.CV cs.GR cs.ROcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

We present LiteReality-Agent, an agentic system for reconstructing real indoor environments as realistic, articulated, and simulation-ready 3D scenes from RGB-D scans. At its core, LiteReality-Agent formulates 3D reconstruction as a coding problem, in which a coding agent gathers evidence using specialised tools and iteratively edits a Python script, Room.py, which can be executed to produce a 3D digital twin of the room. With this formulation, we develop a robust observe-edit-verify harness that supports evidence gathering, measurement, verification, layout optimisation, simulation readiness, and quality control throughout the reconstruction process. LiteReality-Agent produces high-quality reconstructions suitable for simulation and downstream embodied AI tasks. Furthermore, as agent capabilities continue to improve rapidly, the system introduced by LiteReality-Agent remains a strong orchestration framework for future agents: it equips them with specialised tools, structured workflows, and robust verification mechanisms that substantially improve reconstruction quality and reliability. We demonstrate that LiteReality-Agent produces reconstructions that are more geometrically accurate, visually realistic, and simulation-compatible than those generated by recent frontier models, such as Astra and Fable. We therefore view LiteReality-Agent as a practical and important building block for robust real-to-sim systems. Both the source code and the data-capture application are publicly available. Code:https://github.com/LiteReality/LiteReality-Agent/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 55 of its 58 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

55 of 58 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents21 Jul
  2. 2026Articraft: An Agentic System for Scalable Articulated 3D Asset Generation14 May
  3. 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AI10 Feb
  4. 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes9 Feb
  5. 2026Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning16 Jan
  6. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent24 Sep
  7. 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code2 Mar

Other references · 48

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  2. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  3. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  4. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  5. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  6. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  7. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  8. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
  9. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
  10. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  11. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 268
  12. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
36 more
  1. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  2. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  3. 2025Native and Compact Structured Latents for 3D Generationcited by 175
  4. 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 165
  5. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  6. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  7. 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 111
  8. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 106
  9. 2024SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Modelcited by 96
  10. 2024Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Modelcited by 96
  11. 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 96
  12. 20233D-GPT: Procedural 3D Modeling with Large Language Modelscited by 95
  13. 2024Automated Creation of Digital Cousins for Robust Policy Learningcited by 91
  14. 2024Real2Code: Reconstruct Articulated Objects via Code Generationcited by 69
  15. 2020Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrievecited by 65
  16. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  17. 2024The Scene Language: Representing Scenes with Programs, Words, and Embeddingscited by 36
  18. 2025PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policiescited by 33
  19. 2025Decompositional Neural Scene Reconstruction with Generative Diffusion Priorcited by 31
  20. 2025BlenderGym: Benchmarking Foundational Model Systems for Graphics Editingcited by 31
  21. 2025MetaScenes: Towards Automated Replica Creation for Real-world 3D Scanscited by 31
  22. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
  23. 2022Phone2Proc: Bringing Robust Robots Into Our Chaotic Worldcited by 26
  24. 2024Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Videocited by 26
  25. 2025FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placementcited by 25
  26. 2023PSDR-Room: Single Photo to Scene using Differentiable Renderingcited by 23
  27. 2024Re-Thinking Inverse Graphics With Large Language Modelscited by 23
  28. 2022PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenescited by 22
  29. 2024BlenderAlchemy: Editing 3D Graphics with Vision-Language Modelscited by 22
  30. 2025LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scanscited by 21
  31. 2025LL3M: Large Language 3D Modelerscited by 20
  32. 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Videocited by 14
  33. 2026Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Modelscited by 10
  34. 2026Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Controlcited by 4
  35. 2026FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minutecited by 3
  36. 2026HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesiscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumes
  2. 2026Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesis
  3. 2021Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments
  4. 2026UnfoldArt: Zero-Shot Recovery of Full Articulated 3D Objects from Text or Image
  5. 2026MUSE: Agentic 3D Scene Authoring via Memory-Grounded Incremental Requirement Satisfaction
  6. 2026SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenes
  7. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation
  8. 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation