mlsanityData to 14 Oct 2026

FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placement

arXiv 2503.04919 ↗First posted 6 Mar 2025cs.CVcited by 25

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Scene generation with 3D assets presents a complex challenge, requiring both high-level semantic understanding and low-level geometric reasoning. While Multimodal Large Language Models (MLLMs) excel at semantic tasks, their application to 3D scene generation is hindered by their limited grounding on 3D geometry. In this paper, we investigate how to best work with MLLMs in an object placement task. Towards this goal, we introduce a novel framework, FirePlace, that applies existing MLLMs in (1) 3D geometric reasoning and the extraction of relevant geometric details from the 3D scene, (2) constructing and solving geometric constraints on the extracted low-level geometry, and (3) pruning for final placements that conform to common sense. By combining geometric reasoning with real-world understanding of MLLMs, our method can propose object placements that satisfy both geometric constraints as well as high-level semantic common-sense considerations. Our experiments show that these capabilities allow our method to place objects more effectively in complex scenes with intricate geometry, surpassing the quality of prior work.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 50 of its 70 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

50 of 70 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilities1 Jul
  2. 2024Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model27 May
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 47

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  2. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  3. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  4. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  5. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  6. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  7. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  8. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
  9. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  10. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 268
  11. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  12. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
35 more
  1. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 243
  2. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  3. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
  4. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 200
  5. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
  6. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  7. 2020SceneFormer: Indoor Scene Generation with Transformerscited by 158
  8. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  9. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 153
  10. 2024Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generationcited by 137
  11. 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 134
  12. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 106
  13. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  14. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 82
  15. 2023LEGO-Net: Learning Regular Rearrangements of Objects in Roomscited by 76
  16. 2023CityDreamer: Compositional Generative Model of Unbounded 3D Citiescited by 76
  17. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  18. 2021A Systematic Investigation of Commonsense Knowledge in Large Language Modelscited by 56
  19. 2024SemCity: Semantic Scene Generation with Triplane Diffusioncited by 53
  20. 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databasescited by 45
  21. 2023Iterative Motion Editing with Natural Languagecited by 40
  22. 2024Img2CAD: Reverse Engineering 3D CAD Models from Images through VLM-Assisted Conditional Factorizationcited by 35
  23. 2023M3DBench: Let's Instruct Large Models with Multi-modal 3D Promptscited by 32
  24. 20243DMIT: 3D Multi-modal Instruction Tuning for Scene Understandingcited by 32
  25. 2023DiffInDScene: Diffusion-based High-Quality 3D Indoor Scene Generationcited by 23
  26. 2024Re-Thinking Inverse Graphics With Large Language Modelscited by 23
  27. 2024Visual Question Answering Instruction: Unlocking Multimodal Large Language Model To Domain-Specific Visual Multitaskscited by 20
  28. 2024SceneMotifCoder: Example-driven Visual Program Learning for Generating 3D Object Arrangementscited by 17
  29. 2024SceneGPT: A Language Model for 3D Scene Understandingcited by 12
  30. 2024DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D Diffusioncited by 12
  31. 2023Aladdin: Zero-Shot Hallucination of Stylized 3D Assets from Abstract Scene Descriptionscited by 11
  32. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Modelscited by 10
  33. 2024ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generationcited by 9
  34. 2024DreamScape: 3D Scene Creation via Gaussian Splatting joint Correlation Modelingcited by 7
  35. 2024ScalingGaussian: Enhancing 3D Content Creation with Generative Gaussian Splattingcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 18.

25 papers.

Sort:

By area:

  1. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agentcited by 56
  2. 2025HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generationcited by 24
  3. 2025FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representationscited by 13
  4. 2025PlaceIt3D: Language-Guided Object Placement in Real 3D Scenescited by 9
  5. 2025BlenderFusion: 3D-Grounded Visual Editing and Generative Compositingcited by 7
  6. 2025ScanEdit: Hierarchically-Guided Functional 3D Scan Editingcited by 4
  7. 2025VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangementcited by 4
  8. 2026Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editingcited by 3
  9. 2026Mind-of-Director: Multi-modal Agent-Driven Film Previsualization via Collaborative Decision-Makingcited by 2
  10. 2026SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generationcited by 2
  11. 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
  12. 2026THOM: Generating Physically Plausible Hand-Object Meshes From Textcited by 1
13 more
  1. 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenescited by 1
  2. 2026SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenescited by 0
  3. 2026SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptionscited by 0
  4. 2026HiddenObjects: Scalable Diffusion-Distilled Spatial Priors for Object Placementcited by 0
  5. 2026ProcFunc: Function-Oriented Abstractions for Procedural 3D Generation in Pythoncited by 0
  6. 2026LabBuilder: Protocol-Grounded 3D Layout Generation for Interactable and Safe Laboratorycited by 0
  7. 2026OmniLayout: A Schematic-Coupled Multimodal Benchmark for Constraint-Aware Geometric Reasoning in PCB Layoutcited by 0
  8. 2026Learning an Interior Layout Policy in a Domain Specific Language Action Spacecited by 0
  9. 2026Encoded but Not Actionable: Auditing the Decode-Generate-Steer Gap in Frozen LLMs for Geometric Constraintscited by 0
  10. 2026IM-ENGINE: Image Editing for Embodied Data Generationcited by 0
  11. 2026LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstructioncited by 0
  12. 2026SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RLcited by 0
  13. 2026Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Flame3D: Zero-shot Compositional Reasoning of 3D Scenes with Agentic Language Models
  2. 2026Beyond Single Object: Learning 3D Relations with Large Language Models
  3. 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models
  4. 2026Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering
  5. 2026Boosting MLLM Spatial Reasoning with Geometrically Referenced 3D Scene Representations
  6. 2026SSR: Pushing the Limit of Spatial Intelligence with Structured Scene Reasoning
  7. 2025MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation
  8. 2025S2-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance