mlsanityData to 7 Oct 2026

Engine-Native Editable 3D World Reconstruction with Objects and Lighting

arXiv 2607.20889 ↗First posted 23 Jul 2026cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Editable 3D scene creation requires object instances and lights that can be inspected, moved, and imported into standard engines, yet existing single-image methods largely stop at room-scale geometry, baked/global illumination, or text-driven generation. We introduce Lumera (Light-aware Unified Engine-native Reconstruction and Assembly), a benchmark and reference pipeline for engine-native, light-aware 3D scene parsing from a single image. Lumera-2K is built from 2,513 UE5 projects and provides 3.73M components, 63M object instances, 102.6K engine-native parametric lights, and 95.1K camera views. On this data, Lumera-Box and Lumera-Light adapt VLM to parse object boxes and parametric light tuples (x,y,z,r,g,b,I), which are assembled with per-object mesh reconstruction, HDR environment estimation, and a bounded agentic refinement loop. In a sanitized box benchmark against DetAny3D, SpatialLM, N3D-VLM, and WildDet3D, Lumera-Box obtains the strongest overall detection, geometry, semantic, and layout scores (merged mAP 0.1141, IoU-B 0.2472, F-score 0.2762), while WildDet3D remains stronger on anchor recall. For lights, Lumera-Light recovers almost all non-empty scenes (recall 0.998) but remains limited at individual-light localization (F1 0.209 at 0.5 m); matched lights have median position error 0.261 m, median DeltaE2000 4.59, and intensity Pearson r=0.628. These results establish parametric lights as a measurable editable-scene target and expose remaining bottlenecks in relation structure, light recall/intensity, and cross-engine generalization.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 50 of its 82 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

50 of 82 references.

References · 50

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  5. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
  6. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  7. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  8. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  9. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  10. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  11. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  12. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
38 more
  1. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  2. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  3. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  4. 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 266
  5. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  6. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  7. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  8. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  9. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  10. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  11. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  12. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  13. 2023Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenescited by 75
  14. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
  15. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 65
  16. 2025RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assetscited by 47
  17. 2025One Model to Rig Them All: Diverse Skeleton Rigging with UniRigcited by 44
  18. 2025MagicArticulate: Make Your 3D Models Articulation-Readycited by 43
  19. 2025TexVerse: A Universe of 3D Objects with High-Resolution Texturescited by 41
  20. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 41
  21. 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AIcited by 40
  22. 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenescited by 39
  23. 2025Human Motion Video Generation: A Surveycited by 25
  24. 2024DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characterscited by 24
  25. 2024Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic Solidscited by 23
  26. 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
  27. 2021ODAM: Object Detection, Association, and Mapping using Posed RGB Videocited by 22
  28. 2023CAD-Estate: Large-scale CAD Model Annotation in RGB Videoscited by 14
  29. 2024Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputscited by 12
  30. 2025BrepGPT: Autoregressive B-rep Generation with Voronoi Half-Patchcited by 12
  31. 2026WildDet3D: Scaling Promptable 3D Detection in the Wildcited by 12
  32. 2026From Frames to Sequences: Temporally Consistent Human-Centric Dense Predictioncited by 7
  33. 2026HVG-3D: Bridging Real and Simulation Domains for 3D-Conditional Hand-Object Interaction Video Synthesiscited by 7
  34. 2026Animator-Centric Skeleton Generation on Objects with Fine-Grained Detailscited by 7
  35. 2026Feedforward 3D Editing Learns from Semantic-Part Transformationcited by 7
  36. 2026CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generationcited by 6
  37. 2026LottieGPT: Tokenizing Vector Animation for Autoregressive Generationcited by 5
  38. 2025EA3D: Online Open-World 3D Object Extraction from Streaming Videoscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 20263D-CoS: A New 3D Reconstruction Paradigm Based on VLM Code Synthesis
  2. 2026LEGO-Anything: Coding Agents for 3D Scene Reconstruction
  3. 2025IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Rendering
  4. 2026Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Models
  5. 2026Video Generation Models Are Inherent Lighting Estimators
  6. 2025TRELLISWorld: Training-Free World Generation from Object Generators
  7. 2026SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
  8. 2026Training-Free Instance-Aware 3D Scene Reconstruction and Diffusion-Based View Synthesis from Sparse Images