Engine-Native Editable 3D World Reconstruction with Objects and Lighting
Not on the map. Why?
Abstract (arXiv)
Editable 3D scene creation requires object instances and lights that can be inspected, moved, and imported into standard engines, yet existing single-image methods largely stop at room-scale geometry, baked/global illumination, or text-driven generation. We introduce Lumera (Light-aware Unified Engine-native Reconstruction and Assembly), a benchmark and reference pipeline for engine-native, light-aware 3D scene parsing from a single image. Lumera-2K is built from 2,513 UE5 projects and provides 3.73M components, 63M object instances, 102.6K engine-native parametric lights, and 95.1K camera views. On this data, Lumera-Box and Lumera-Light adapt VLM to parse object boxes and parametric light tuples (x,y,z,r,g,b,I), which are assembled with per-object mesh reconstruction, HDR environment estimation, and a bounded agentic refinement loop. In a sanitized box benchmark against DetAny3D, SpatialLM, N3D-VLM, and WildDet3D, Lumera-Box obtains the strongest overall detection, geometry, semantic, and layout scores (merged mAP 0.1141, IoU-B 0.2472, F-score 0.2762), while WildDet3D remains stronger on anchor recall. For lights, Lumera-Light recovers almost all non-empty scenes (recall 0.998) but remains limited at individual-light localization (F1 0.209 at 0.5 m); matched lights have median position error 0.261 m, median DeltaE2000 4.59, and intensity Pearson r=0.628. These results establish parametric lights as a measurable editable-scene target and expose remaining bottlenecks in relation structure, light recall/intensity, and cross-engine generalization.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 50 of its 82 references that are in the map. First posted on arXiv 23 Jul 2026.
Builds on
50 of 82 references.
References · 50
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
- 2024Grounding Image Matching in 3D with MASt3Rcited by 960
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
38 more
- 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
- 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
- 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
- 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 266
- 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
- 2025SAM 3D: 3Dfy Anything in Imagescited by 230
- 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
- 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
- 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
- 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
- 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
- 2023Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenescited by 75
- 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
- 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 65
- 2025RigAnything: Template-Free Autoregressive Rigging for Diverse 3D Assetscited by 47
- 2025One Model to Rig Them All: Diverse Skeleton Rigging with UniRigcited by 44
- 2025MagicArticulate: Make Your 3D Models Articulation-Readycited by 43
- 2025TexVerse: A Universe of 3D Objects with High-Resolution Texturescited by 41
- 2025Puppeteer: Rig and Animate Your 3D Modelscited by 41
- 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AIcited by 40
- 2026SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenescited by 39
- 2025Human Motion Video Generation: A Surveycited by 25
- 2024DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characterscited by 24
- 2024Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic Solidscited by 23
- 2025Generating Physically Stable and Buildable Brick Structures from Textcited by 23
- 2021ODAM: Object Detection, Association, and Mapping using Posed RGB Videocited by 22
- 2023CAD-Estate: Large-scale CAD Model Annotation in RGB Videoscited by 14
- 2024Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputscited by 12
- 2025BrepGPT: Autoregressive B-rep Generation with Voronoi Half-Patchcited by 12
- 2026WildDet3D: Scaling Promptable 3D Detection in the Wildcited by 12
- 2026From Frames to Sequences: Temporally Consistent Human-Centric Dense Predictioncited by 7
- 2026HVG-3D: Bridging Real and Simulation Domains for 3D-Conditional Hand-Object Interaction Video Synthesiscited by 7
- 2026Animator-Centric Skeleton Generation on Objects with Fine-Grained Detailscited by 7
- 2026Feedforward 3D Editing Learns from Semantic-Part Transformationcited by 7
- 2026CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generationcited by 6
- 2026LottieGPT: Tokenizing Vector Animation for Autoregressive Generationcited by 5
- 2025EA3D: Online Open-World 3D Object Extraction from Streaming Videoscited by 4
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 20263D-CoS: A New 3D Reconstruction Paradigm Based on VLM Code Synthesis
- 2026LEGO-Anything: Coding Agents for 3D Scene Reconstruction
- 2025IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Rendering
- 2026Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Models
- 2026Video Generation Models Are Inherent Lighting Estimators
- 2025TRELLISWorld: Training-Free World Generation from Object Generators
- 2026SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
- 2026Training-Free Instance-Aware 3D Scene Reconstruction and Diffusion-Based View Synthesis from Sparse Images