FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minute
Not on the map. Why?
Abstract (arXiv)
We present FIRE3D, a unified framework that transforms a single RGB image or casual video into interactable 3D scene assets for games and interactive applications in under a minute for up to twelve textured instances including preprocessing. At the core of FIRE3D is a feed-forward inference pipeline that predicts a compositional scene representation from posed RGB-D observations estimated from the RGB capture, including the 6-DoF pose, bounding box, mesh, and texture for detected objects. By modeling the scene as a collection of discrete entities, FIRE3D produces amodal object assets that can be independently edited and used in interactive applications. Our framework requires no test-time optimization, runs substantially faster than the compared reconstruction systems, and provides object-level completeness beyond existing feed-forward 3D approaches. We demonstrate leading detection accuracy, strong geometry reconstruction, and competitive rendering quality on the evaluated benchmarks, with substantial runtime gains.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 71 of its 78 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
71 of 78 references.
References · 71
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
- 2025DINOv3cited by 1,414
- 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,076
- 2024Grounding Image Matching in 3D with MASt3Rcited by 983
38 more
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
- 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
- 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
- 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
- 2025Continuous 3D Perception Model with Persistent Statecited by 548
- 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
- 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 397
- 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
- 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
- 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
- 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
- 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 356
- 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 319
- 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
- 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 303
- 2020Neural Scene Graphs for Dynamic Scenescited by 298
- 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 295
- 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
- 2025SAM 3D: 3Dfy Anything in Imagescited by 247
- 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 232
- 2023BakedSDF: Meshing Neural SDFs for Real-Time View Synthesiscited by 221
- 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
- 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
- 2022Neural 3D Scene Reconstruction with the Manhattan-world Assumptioncited by 123
- 2021Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Biascited by 118
- 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 117
- 2024SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Modelcited by 96
- 2022Object-Compositional Neural Implicit Surfacescited by 74
- 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 61
- 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 54
- 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AIcited by 50
50 most cited of 71 All 71 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumescited by 0
- 2026LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstructioncited by 0
- 2026LIVIN: Benchmarking Spatial and Embodied Intelligence in Digital Twins of Lived-In Homescited by 0
- 2026LIVIN: Benchmarking Spatial and Embodied Intelligence in Digital Twins of Lived-In Homes8 Oct 2026
- 2026LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstruction1 Oct 2026
- 2026CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumes28 Sep 2026