# CAD-Prompted Generative Models: A Pathway to Feasible and Novel Engineering Designs

arXiv: https://arxiv.org/abs/2407.08675 · HTML: https://mlsanity.com/paper/2407.08675/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 11 Jul 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 15 of 15)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning](https://mlsanity.com/paper/2108.03353/) (2021)
- [The Creativity of Text-to-Image Generation](https://mlsanity.com/paper/2206.02904/) (2022)
- [Large-scale Text-to-Image Generation Models for Visual Artists' Creative Works](https://mlsanity.com/paper/2210.08477/) (2022)
- [3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows](https://mlsanity.com/paper/2210.11603/) (2022)
- [Multi-modal Machine Learning in Engineering Design: A Review and Future Directions](https://mlsanity.com/paper/2302.10909/) (2023)
- [Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design](https://mlsanity.com/paper/2302.02913/) (2023)
- [Using Text-to-Image Generation for Architectural Design Ideation](https://mlsanity.com/paper/2304.10182/) (2023)
- [Constraining Generative Models for Engineering Design with Negative Data](https://mlsanity.com/paper/2306.15166/) (2023)
- [Towards Goal, Feasibility, and Diversity-Oriented Deep Generative Models in Design](https://mlsanity.com/paper/2206.07170/) (2022)

## Built on by (most cited first, 3 of 3)

- [DesignWeaver: Dimensional Scaffolding for Text-to-Image Product Design](https://mlsanity.com/paper/2502.09867/) (2025)
- [Evaluation of Architectural Synthesis Using Generative AI](https://mlsanity.com/paper/2503.02861/) (2025)
- [Parametric-ControlNet: Multimodal Control in Foundation Models for Precise Engineering Design Synthesis](https://mlsanity.com/paper/2412.04707/) (2024)

## Abstract (arXiv)

Text-to-image generative models have increasingly been used to assist designers during concept generation in various creative domains, such as graphic design, user interface design, and fashion design. However, their applications in engineering design remain limited due to the models' challenges in generating images of feasible designs concepts. To address this issue, this paper introduces a method that improves the design feasibility by prompting the generation with feasible CAD images. In this work, the usefulness of this method is investigated through a case study with a bike design task using an off-the-shelf text-to-image model, Stable Diffusion 2.1. A diverse set of bike designs are produced in seven different generation settings with varying CAD image prompting weights, and these designs are evaluated on their perceived feasibility and novelty. Results demonstrate that the CAD image prompting successfully helps text-to-image models like Stable Diffusion 2.1 create visibly more feasible design images. While a general tradeoff is observed between feasibility and novelty, when the prompting weight is kept low around 0.35, the design feasibility is significantly improved while its novelty remains on par with those generated by text prompts alone. The insights from this case study offer some guidelines for selecting the appropriate CAD image prompting weight for different stages of the engineering design process. When utilized effectively, our CAD image prompting method opens doors to a wider range of applications of text-to-image models in engineering design.