# SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass

arXiv: https://arxiv.org/abs/2508.15769 · HTML: https://mlsanity.com/paper/2508.15769/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 75 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 30 of 66)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Structured 3D Latents for Scalable and Versatile 3D Generation](https://mlsanity.com/paper/2412.01506/) (2024)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets](https://mlsanity.com/paper/2406.13897/) (2024)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model](https://mlsanity.com/paper/2311.06214/) (2023)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [LayoutGPT: Compositional Visual Planning and Generation with Large Language Models](https://mlsanity.com/paper/2305.15393/) (2023)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation](https://mlsanity.com/paper/2212.04493/) (2022)
- [Holodeck: Language Guided Generation of 3D Embodied AI Environments](https://mlsanity.com/paper/2312.09067/) (2023)
- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)

## Built on by (most cited first, 30 of 55)

- [ShapeR: Robust Conditional 3D Shape Generation from Casual Captures](https://mlsanity.com/paper/2601.11514/) (2026)
- [WorldGen: From Text to Traversable and Interactive 3D Worlds](https://mlsanity.com/paper/2511.16825/) (2025)
- [SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model](https://mlsanity.com/paper/2512.10957/) (2025)
- [SimRecon: SimReady Compositional Scene Reconstruction from Real Videos](https://mlsanity.com/paper/2603.02133/) (2026)
- [3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image](https://mlsanity.com/paper/2604.04406/) (2026)
- [TRELLISWorld: Training-Free World Generation from Object Generators](https://mlsanity.com/paper/2510.23880/) (2025)
- [One Video, One World: Turning Monocular Video into Physical 4D Scenes](https://mlsanity.com/paper/2606.31388/) (2026)
- [GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence](https://mlsanity.com/paper/2511.21945/) (2025)
- [I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learners](https://mlsanity.com/paper/2512.13683/) (2025)
- [REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image](https://mlsanity.com/paper/2605.30338/) (2026)
- [ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editing](https://mlsanity.com/paper/2509.23607/) (2025)
- [MoCA: Mixture-of-Components Attention for Scalable Compositional 3D Generation](https://mlsanity.com/paper/2512.07628/) (2025)
- [SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image](https://mlsanity.com/paper/2606.03994/) (2026)
- [Points-to-3D: Structure-Aware 3D Generation with Point Cloud Priors](https://mlsanity.com/paper/2603.18782/) (2026)
- [PhysOmni: Physics-Grounded Multi-Object Scene Generation from a Single Image with Real-Time Interaction](https://mlsanity.com/paper/2605.20290/) (2026)
- [IL3D: A Large-Scale Indoor Layout Dataset for LLM-Driven 3D Scene Generation](https://mlsanity.com/paper/2510.12095/) (2025)
- [TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation](https://mlsanity.com/paper/2512.01204/) (2025)
- [Rein3D: Reinforced 3D Indoor Scene Generation with Panoramic Video Diffusion Models](https://mlsanity.com/paper/2604.10578/) (2026)
- [ReplicateAnyScene: Zero-Shot Video-to-3D Composition via Textual-Visual-Spatial Alignment](https://mlsanity.com/paper/2604.10789/) (2026)
- [φ-Scene: Physically Grounded Image-to-3D Scene Reconstruction](https://mlsanity.com/paper/2606.21596/) (2026)
- [FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minute](https://mlsanity.com/paper/2609.08848/) (2026)
- [Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image](https://mlsanity.com/paper/2603.05908/) (2026)
- [Mind-of-Director: Multi-modal Agent-Driven Film Previsualization via Collaborative Decision-Making](https://mlsanity.com/paper/2603.14790/) (2026)
- [JRM: Joint Reconstruction Model for Multiple Objects without Alignment](https://mlsanity.com/paper/2603.25985/) (2026)
- [PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization](https://mlsanity.com/paper/2604.10125/) (2026)
- [WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes](https://mlsanity.com/paper/2605.15843/) (2026)
- [SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestration](https://mlsanity.com/paper/2606.08402/) (2026)
- [Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling](https://mlsanity.com/paper/2608.30821/) (2026)
- [WorldSculpt: Generating Compositional Worlds from Grounded Videos](https://mlsanity.com/paper/2609.05416/) (2026)
- [Inferring Compositional 4D Scenes without Ever Seeing One](https://mlsanity.com/paper/2512.05272/) (2025)

## Abstract (arXiv)

3D content generation has recently attracted significant research interest, driven by its critical applications in VR/AR and embodied AI. In this work, we tackle the challenging task of synthesizing multiple 3D assets within a single scene image. Concretely, our contributions are fourfold: (i) we present SceneGen, a novel framework that takes a scene image and corresponding object masks as input, simultaneously producing multiple 3D assets with geometry and texture. Notably, SceneGen operates with no need for extra optimization or asset retrieval; (ii) we introduce a novel feature aggregation module that integrates local and global scene information from visual and geometric encoders within the feature extraction module. Coupled with a position head, this enables the generation of 3D assets and their relative spatial positions in a single feedforward pass; (iii) we demonstrate SceneGen's direct extensibility to multi-image input scenarios. Despite being trained solely on single-image inputs, our architecture yields improved generation performance when multiple images are provided; and (iv) extensive quantitative and qualitative evaluations confirm the efficiency and robustness of our approach. We believe this paradigm offers a novel solution for high-quality 3D content generation, potentially advancing its practical applications in downstream tasks. The code and model will be publicly available at: https://mengmouxu.github.io/SceneGen.