mlsanityData to 7 Oct 2026

Sketch2Prototype: Rapid Conceptual Design Exploration and Prototyping with Generative AI

arXiv 2405.12985 ↗First posted 26 Mar 2024cs.HC cs.AI cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Sketch2Prototype is an AI-based framework that transforms a hand-drawn sketch into a diverse set of 2D images and 3D prototypes through sketch-to-text, text-to-image, and image-to-3D stages. This framework, shown across various sketches, rapidly generates text, image, and 3D modalities for enhanced early-stage design exploration. We show that using text as an intermediate modality outperforms direct sketch-to-3D baselines for generating diverse and manufacturable 3D models. We find limitations in current image-to-3D techniques, while noting the value of the text modality for user-feedback and iterative design augmentation.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 16 of its 39 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

16 of 39 references.

References · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  9. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  10. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
4 more
  1. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  2. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  3. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  4. 2023From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Designcited by 51

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024Query2CAD: Generating CAD models using natural language queriescited by 28
  2. 2025AI Agents in Engineering Design: A Multi-Agent Framework for Aesthetic and Aerodynamic Car Designcited by 20
  3. 2026CADSmith: Multi-Agent CAD Generation with Programmatic Geometric Validationcited by 11
  4. 2024Speech to Reality: On-Demand Production using Natural Language, 3D Generative AI, and Discrete Robotic Assemblycited by 4
  5. 2025Reimagining Personal Data: Unlocking the Potential of AI-Generated Images in Personal Data Meaning-Makingcited by 2
  6. 2025Evaluation of Architectural Synthesis Using Generative AIcited by 1
  7. 2025Toward Knowledge-Guided AI for Inverse Design in Manufacturing: A Perspective on Domain, Physics, and Human-AI Synergycited by 1
  8. 2025Text to Robotic Assembly of Multi Component Objects using 3D Generative AI and Vision Language Modelscited by 1
  9. 2026Investigating Social Bias in Narrative Image Generationcited by 1
  10. 2024Creativity in the Age of AI: Evaluating the Impact of Generative AI on Design Outputs and Designers' Creative Thinkingcited by 0
  11. 2025PartStickers: Generating Parts of Objects for Rapid Prototypingcited by 0
  12. 2026Agentic AI in Engineering and Manufacturing: Industry Perspectives on Utility, Adoption, Challenges, and Opportunitiescited by 0
1 more
  1. 2026SemanticSlider3D: Training-Free Continuous Semantic Editing for 3D Objectscited by 0