# From Flat to Spatial: Comparison of 4 methods constructing 3D, 2 and 1/2D Models from 2D Plans with neural networks

arXiv: https://arxiv.org/abs/2407.19970 · HTML: https://mlsanity.com/paper/2407.19970/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 23 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 21 of 21)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model](https://mlsanity.com/paper/2311.06214/) (2023)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [Text-to-3D using Gaussian Splatting](https://mlsanity.com/paper/2309.16585/) (2023)
- [One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion](https://mlsanity.com/paper/2311.07885/) (2023)
- [Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers](https://mlsanity.com/paper/2312.09147/) (2023)
- [GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models](https://mlsanity.com/paper/2310.08529/) (2023)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer](https://mlsanity.com/paper/2405.14832/) (2024)
- [NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views](https://mlsanity.com/paper/2211.16431/) (2022)
- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [Text-to-3D with Classifier Score Distillation](https://mlsanity.com/paper/2310.19415/) (2023)
- [HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance](https://mlsanity.com/paper/2305.18766/) (2023)
- [Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks](https://mlsanity.com/paper/1909.00321/) (2019)
- [IT3D: Improved Text-to-3D Generation with Explicit View Synthesis](https://mlsanity.com/paper/2308.11473/) (2023)
- [MVD2: Efficient Multiview 3D Reconstruction for Multiview Diffusion](https://mlsanity.com/paper/2402.14253/) (2024)

## Abstract (arXiv)

In the field of architecture, the conversion of single images into 2 and 1/2D and 3D meshes is a promising technology that enhances design visualization and efficiency. This paper evaluates four innovative methods: "One-2-3-45," "CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model," "Instant Mesh," and "Image-to-Mesh." These methods are at the forefront of this technology, focusing on their applicability in architectural design and visualization. They streamline the creation of 3D architectural models, enabling rapid prototyping and detailed visualization from minimal initial inputs, such as photographs or simple sketches.One-2-3-45 leverages a diffusion-based approach to generate multi-view reconstructions, ensuring high geometric fidelity and texture quality. CRM utilizes a convolutional network to integrate geometric priors into its architecture, producing detailed and textured meshes quickly and efficiently. Instant Mesh combines the strengths of multi-view diffusion and sparse-view models to offer speed and scalability, suitable for diverse architectural projects. Image-to-Mesh leverages a generative adversarial network (GAN) to produce 3D meshes from single images, focusing on maintaining high texture fidelity and geometric accuracy by incorporating image and depth map data into its training process. It uses a hybrid approach that combines voxel-based representations with surface reconstruction techniques to ensure detailed and realistic 3D models.This comparative study highlights each method's contribution to reducing design cycle times, improving accuracy, and enabling flexible adaptations to various architectural styles and requirements. By providing architects with powerful tools for rapid visualization and iteration, these advancements in 3D mesh generation are set to revolutionize architectural practices.