# SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesis

arXiv: https://arxiv.org/abs/2503.14756 · HTML: https://mlsanity.com/paper/2503.14756/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 100 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Builds on (most cited first, 30 of 42)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [LayoutGPT: Compositional Visual Planning and Generation with Large Language Models](https://mlsanity.com/paper/2305.15393/) (2023)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Long-CLIP: Unlocking the Long-Text Capability of CLIP](https://mlsanity.com/paper/2403.15378/) (2024)
- [Holodeck: Language Guided Generation of 3D Embodied AI Environments](https://mlsanity.com/paper/2312.09067/) (2023)
- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis](https://mlsanity.com/paper/2303.14207/) (2023)
- [GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation](https://mlsanity.com/paper/2401.04092/) (2024)
- [Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation](https://mlsanity.com/paper/2406.11824/) (2024)
- [SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code](https://mlsanity.com/paper/2403.01248/) (2024)
- [PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI](https://mlsanity.com/paper/2404.09465/) (2024)
- [InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior](https://mlsanity.com/paper/2402.04717/) (2024)
- [Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Models](https://mlsanity.com/paper/1811.12463/) (2018)
- [LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models](https://mlsanity.com/paper/2412.02193/) (2024)
- [I-Design: Personalized LLM Interior Designer](https://mlsanity.com/paper/2404.02838/) (2024)

## Built on by (most cited first, 26 of 26)

- [HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generation](https://mlsanity.com/paper/2503.16848/) (2025)
- [SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects](https://mlsanity.com/paper/2605.19587/) (2026)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis](https://mlsanity.com/paper/2508.18597/) (2025)
- [SceneNAT: Masked Generative Modeling for Language-Guided Indoor Scene Synthesis](https://mlsanity.com/paper/2601.07218/) (2026)
- [PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimization](https://mlsanity.com/paper/2604.10125/) (2026)
- [SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis](https://mlsanity.com/paper/2604.13035/) (2026)
- [ShellMaker: Language-Guided Exterior Completion under Structural Constraints](https://mlsanity.com/paper/2606.31680/) (2026)
- [Beyond Placement and Articulation: Usage-Driven Code Scenes for Embodied Interaction](https://mlsanity.com/paper/2608.18840/) (2026)
- [MLFM: Multi-Layered Feature Maps for Richer Language Understanding in Zero-Shot Semantic Navigation](https://mlsanity.com/paper/2507.07299/) (2025)
- [Apples on the Table? Evaluating Text-Guided 3D Scene Synthesis via Fine-Grained Constraint Verification](https://mlsanity.com/paper/2511.03001/) (2025)
- [HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models](https://mlsanity.com/paper/2604.10772/) (2026)
- [Function2Scene: 3D Indoor Scene Layout from Functional Specifications](https://mlsanity.com/paper/2605.30819/) (2026)
- [FuncRoom-Agent: Sequential Feed-Forward 3D Functional Indoor Scene Generation](https://mlsanity.com/paper/2608.29519/) (2026)
- [Search-to-World: Evaluation of 3D World Delivery from User Request through Web Search](https://mlsanity.com/paper/2609.07605/) (2026)
- [Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes](https://mlsanity.com/paper/2609.36777/) (2026)
- [SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenes](https://mlsanity.com/paper/2603.29798/) (2026)
- [Orchestrating Spatial Semantics via a Zone-Graph Paradigm for Intricate Indoor Scene Generation](https://mlsanity.com/paper/2605.02537/) (2026)
- [Architect-Ant: Editable Automatic Furnishing of Architectural Floor Plans](https://mlsanity.com/paper/2606.10953/) (2026)
- [Text-Driven 3D Indoor Scene Synthesis in Non-Manhattan Environments](https://mlsanity.com/paper/2607.02407/) (2026)
- [Roomer: Reflective Object-Grounded Model Editing and Repair for 3D Indoor Layout Synthesis](https://mlsanity.com/paper/2608.01973/) (2026)
- [Learning an Interior Layout Policy in a Domain Specific Language Action Space](https://mlsanity.com/paper/2608.07547/) (2026)
- [SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution](https://mlsanity.com/paper/2609.05594/) (2026)
- [HappyWorld-Bench](https://mlsanity.com/paper/2609.24308/) (2026)
- [Frozen Scenes, Shifting Winners: Configuration Fragility in Text-to-3D Evaluation](https://mlsanity.com/paper/2610.00447/) (2026)
- [WorldBench: Evaluating LLMs on Three.js Voxel World Generation](https://mlsanity.com/paper/2610.10622/) (2026)

## Abstract (arXiv)

Despite recent advances in text-conditioned 3D indoor scene generation, there remain gaps in the evaluation of these methods. Existing metrics often measure realism by comparing generated scenes to a set of ground-truth scenes, but they overlook how well scenes follow the input text and capture implicit expectations of plausibility. We present SceneEval, an evaluation framework designed to address these limitations. SceneEval introduces fine-grained metrics for explicit user requirements-including object counts, attributes, and spatial relationships-and complementary metrics for implicit expectations such as support, collisions, and navigability. Together, these provide interpretable and comprehensive assessments of scene quality. To ground evaluation, we curate SceneEval-500, a benchmark of 500 text descriptions with detailed annotations of expected scene properties. This dataset establishes a common reference for reproducible and systematic comparison across scene generation methods. We evaluate six recent scene generation approaches using SceneEval and demonstrate its ability to provide detailed assessments of the generated scenes, highlighting strengths and areas for improvement across multiple dimensions. Our results identify significant gaps in current methods, underscoring the need for further research toward practical and controllable scene synthesis.