mlsanityData to 14 Oct 2026

3D Software Synthesis Driven by Constraint-Expressive Intermediate Representation

arXiv 2507.18625 ↗First posted 24 Jul 2025cs.CV cs.AI cs.MMcited by 4

Not on the map. Why?

Abstract (arXiv)

Graphical user interface (UI) software has undergone a fundamental transformation from traditional two-dimensional (2D) desktop/web/mobile interfaces to spatial three-dimensional (3D) environments. While existing work has made remarkable success in automated 2D software generation, such as HTML/CSS and mobile app interface code synthesis, the generation of 3D software still remains under-explored. Current methods for 3D software generation usually generate the 3D environments as a whole and cannot modify or control specific elements in the software. Furthermore, these methods struggle to handle the complex spatial and semantic constraints inherent in the real world. To address the challenges, we present Scenethesis, a novel requirement-sensitive 3D software synthesis approach that maintains formal traceability between user specifications and generated 3D software. Scenethesis is built upon ScenethesisLang, a domain-specific language that serves as a granular constraint-aware intermediate representation (IR) to bridge natural language requirements and executable 3D software. It serves both as a comprehensive scene description language enabling fine-grained modification of 3D software elements and as a formal constraint-expressive specification language capable of expressing complex spatial constraints. By decomposing 3D software synthesis into stages operating on ScenethesisLang, Scenethesis enables independent verification, targeted modification, and systematic constraint satisfaction. Our evaluation demonstrates that Scenethesis accurately captures over 80% of user requirements and satisfies more than 90% of hard constraints while handling over 100 constraints simultaneously. Furthermore, Scenethesis achieves a 42.8% improvement in BLIP-2 visual evaluation scores compared to the state-of-the-art method.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 41 of its 93 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

41 of 93 references.

References · 41

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  3. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  4. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
  5. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  6. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  7. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 268
  8. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 172
  9. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 166
  10. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
  11. 2023GPT-4V(ision) as a Generalist Evaluator for Vision-Language Taskscited by 119
  12. 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 117
29 more
  1. 2024GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splattingcited by 109
  2. 2023GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphscited by 97
  3. 2024Automated Creation of Digital Cousins for Robust Policy Learningcited by 91
  4. 2024I-Design: Personalized LLM Interior Designercited by 87
  5. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 82
  6. 2023LEGO-Net: Learning Regular Rearrangements of Objects in Roomscited by 76
  7. 2024DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Samplingcited by 72
  8. 2018Holistic 3D Scene Parsing and Reconstruction from a Single RGB Imagecited by 69
  9. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 67
  10. 2023CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusioncited by 54
  11. 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototypingcited by 51
  12. 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databasescited by 45
  13. 2025DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generationcited by 45
  14. 2021Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Explorationcited by 44
  15. 2024WebCode2M: A Real-World Dataset for Code Generation from Webpage Designscited by 44
  16. 2024SceneTeller: Language-to-3D Scene Generationcited by 43
  17. 2021Indoor Scene Generation from a Collection of Semantic-Segmented Depth Imagescited by 27
  18. 2017Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammarscited by 25
  19. 2024Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approachcited by 24
  20. 2024LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systemscited by 21
  21. 2025MLLM-Based UI2Code Automation Guided by UI Layout Informationcited by 17
  22. 2023CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embeddingcited by 16
  23. 2017SceneSuggest: Context-driven 3D Scene Designcited by 8
  24. 2023Testing learning-enabled cyber-physical systems with Large-Language Models: A Formal Approachcited by 5
  25. 2023OpenSBT: A Modular Framework for Search-based Testing of Automated Driving Systemscited by 4
  26. 2023Practical Non-Intrusive GUI Exploration Testing with Visual-based Robotic Armscited by 3
  27. 2022Towards Build Verifiability for Java-based Systemscited by 2
  28. 2023Safe-DS: A Domain Specific Language to Make Data Science Safecited by 2
  29. 2024In-Simulation Testing of Deep Learning Vision Models in Autonomous Robotic Manipulatorscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026MAGIC: Transition-Aware Generation of Navigable Multi-Scene Game Worlds with Large Language Modelscited by 1
  2. 2025Runtime Failure Hunting for Physics Engine Based Software Systems: How Far Can We Go?cited by 0
  3. 2026Frozen Scenes, Shifting Winners: Configuration Fragility in Text-to-3D Evaluationcited by 0
  4. 2026VIGOR: Verifiable Industrial Geometry Optimization and Repair in Scene Programscited by 0