HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation
Not on the map. Why?
Abstract (arXiv)
Scene-level 3D generation represents a critical frontier in multimedia and computer graphics, yet existing approaches either suffer from limited object categories or lack editing flexibility for interactive applications. In this paper, we present HiScene, a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation and delivers high-fidelity scenes with compositional identities and aesthetic scene content. Our key insight is treating scenes as hierarchical "objects" under isometric views, where a room functions as a complex object that can be further decomposed into manipulatable items. This hierarchical approach enables us to generate 3D content that aligns with 2D representations while maintaining compositional structure. To ensure completeness and spatial alignment of each decomposed instance, we develop a video-diffusion-based amodal completion technique that effectively handles occlusions and shadows between objects, and introduce shape prior injection to ensure spatial coherence within the scene. Experimental results demonstrate that our method produces more natural object arrangements and complete object instances suitable for interactive applications, while maintaining physical plausibility and alignment with user inputs.
Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 90 of its 109 references that are in the map. First posted on arXiv 17 Apr 2025.
Builds on
90 of 109 references.
References · 90
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Segment Anythingcited by 8,158
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
38 more
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
- 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
- 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
- 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
- 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
- 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
- 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Modelscited by 449
- 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
- 2023Emergent Correspondence from Image Diffusioncited by 437
- 2022Blended Latent Diffusioncited by 430
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
- 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 387
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
- 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
- 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
- 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
- 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
- 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
- 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
- 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
- 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
- 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 219
- 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 200
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
- 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
- 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 159
- 2023WonderJourney: Going from Anywhere to Everywherecited by 157
- 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fieldscited by 122
- 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 120
50 most cited of 90 All 90 (JSON)
Built on by
Citing papers per year: 2025: 6; 2026: 12.
18 papers.
Sort:
By area:
- 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
- 20253D Scene Generation: A Surveycited by 24
- 2025CUPID: Generative 3D Reconstruction via Joint Object and Pose Modelingcited by 8
- 2026PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangementcited by 7
- 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 6
- 2025HLG: Comprehensive 3D Room Construction via Hierarchical Layout Generationcited by 4
- 2025ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editingcited by 4
- 2026MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphingcited by 4
- 2026SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Imagecited by 4
- 2025GeoWorld: Providing Full-frame Geometry Features to Facilitate 3D Scene Generationcited by 3
- 2026PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimizationcited by 2
- 2026WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenescited by 2
6 more
- 2026SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestrationcited by 2
- 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
- 2026Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Imagecited by 1
- 2026HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesiscited by 1
- 2026InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360deg Imagecited by 0
- 2026HIVE-3D: Hierarchical Voxel Enhancement for High-Quality 3D Scene Generationcited by 0
- 2026HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis22 Sep 2026
- 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling31 Aug 2026
- 2026WorldClaw: Agentic 3D Open-World Generation at Scale5 Aug 2026
- 2026HIVE-3D: Hierarchical Voxel Enhancement for High-Quality 3D Scene Generation15 Jul 2026
- 2026InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360deg Image4 Jul 2026
- 2026SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestration7 Jun 2026
- 2026SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image2 Jun 2026
- 2026WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes15 May 2026
- 2026PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization11 Apr 2026
- 2026Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image6 Mar 2026
- 2026PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangement16 Feb 2026
- 2026MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphing1 Jan 2026
6 more
- 2025GeoWorld: Providing Full-frame Geometry Features to Facilitate 3D Scene Generation28 Nov 2025
- 2025CUPID: Generative 3D Reconstruction via Joint Object and Pose Modeling23 Oct 2025
- 2025ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editing28 Sep 2025
- 2025HLG: Comprehensive 3D Room Construction via Hierarchical Layout Generation25 Aug 2025
- 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass21 Aug 2025
- 20253D Scene Generation: A Survey8 May 2025
- 20253D Scene Generation: A Surveycited by 24
- 20253D Scene Generation: A Survey8 May 2025
- 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 49
- 2025CUPID: Generative 3D Reconstruction via Joint Object and Pose Modelingcited by 8
- 2025HLG: Comprehensive 3D Room Construction via Hierarchical Layout Generationcited by 4
- 2025ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editingcited by 4
- 2026MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphingcited by 4
- 2026SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Imagecited by 4
- 2025GeoWorld: Providing Full-frame Geometry Features to Facilitate 3D Scene Generationcited by 3
- 2026PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimizationcited by 2
- 2026SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestrationcited by 2
- 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
- 2026Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Imagecited by 1
- 2026HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesiscited by 1
12 most cited of 14 All 14 (JSON)
- 2026HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis22 Sep 2026
- 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling31 Aug 2026
- 2026HIVE-3D: Hierarchical Voxel Enhancement for High-Quality 3D Scene Generation15 Jul 2026
- 2026InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360deg Image4 Jul 2026
- 2026SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestration7 Jun 2026
- 2026SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image2 Jun 2026
- 2026PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization11 Apr 2026
- 2026Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image6 Mar 2026
- 2026MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphing1 Jan 2026
- 2025GeoWorld: Providing Full-frame Geometry Features to Facilitate 3D Scene Generation28 Nov 2025
- 2025CUPID: Generative 3D Reconstruction via Joint Object and Pose Modeling23 Oct 2025
- 2025ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editing28 Sep 2025
12 newest of 14 All 14 (JSON)