# Synthetic Homes: A Multimodal Generative AI Pipeline for Residential Building Data Generation under Data Scarcity

arXiv: https://arxiv.org/abs/2509.09794 · HTML: https://mlsanity.com/paper/2509.09794/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 69 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token](https://mlsanity.com/paper/2501.03895/) (2025)
- [SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Data](https://mlsanity.com/paper/2404.15821/) (2024)
- [StructuredRAG: JSON Response Formatting with Large Language Models](https://mlsanity.com/paper/2408.11061/) (2024)
- [Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detection](https://mlsanity.com/paper/2205.03302/) (2022)
- [Systematic Assessment of Tabular Data Synthesis](https://mlsanity.com/paper/2402.06806/) (2024)
- [Leveraging Generative AI for Urban Digital Twins: A Scoping Review on the Autonomous Generation of Urban Data, Scenarios, Designs, and 3D City Models for Smart City Advancement](https://mlsanity.com/paper/2405.19464/) (2024)
- [SolarGAN: Synthetic Annual Solar Irradiance Time Series on Urban Building Facades via Deep Generative Networks](https://mlsanity.com/paper/2206.00747/) (2022)
- [Automatic digital twin data model generation of building energy systems from piping and instrumentation diagrams](https://mlsanity.com/paper/2108.13912/) (2021)

## Abstract (arXiv)

Computational models have emerged as powerful tools for multi-scale energy modeling research at the building and urban scale, supporting data-driven analysis across building and urban energy systems. However, these models require large amounts of building parameter data that is often inaccessible, expensive to collect, or subject to privacy constraints. We introduce a modular framework that applies generative Artificial Intelligence (AI) to construct simulation-ready building datasets from publicly available records and imagery. To improve the reliability of this framework, we evaluate both its AI components and its overall result. Our occlusion analysis demonstrates that for our selected images, LLaVA achieves greater visual focus than a GPT-based alternative for building image processing. We also assess plausibility of our results against a national reference dataset, finding that our synthetic data overlaps more than 95% for three of the four selected variables. This work aims to reduce dependence on costly or restricted data sources, lowering barriers to building-scale energy research and Machine Learning (ML)-driven urban energy modeling, thereby providing simulation-ready datasets intended to support downstream applications such as energy modeling, retrofit analysis, and urban-scale simulation under data scarcity.