mlsanityData to 7 Oct 2026

Speech to Reality: On-Demand Production using Natural Language, 3D Generative AI, and Discrete Robotic Assembly

arXiv 2409.18390 ↗First posted 27 Sep 2024cs.RO cs.AI cs.HCcited by 4

Not on the map. Why?

Abstract (arXiv)

We present a system that transforms speech into physical objects using 3D generative AI and discrete robotic assembly. By leveraging natural language, the system makes design and manufacturing more accessible to people without expertise in 3D modeling or robotic programming. While generative AI models can produce a wide range of 3D meshes, AI-generated meshes are not directly suitable for robotic assembly or account for fabrication constraints. To address this, we contribute a workflow that integrates natural language, 3D generative AI, geometric processing, and discrete robotic assembly. The system discretizes the AI-generated geometry and modifies it to meet fabrication constraints such as component count, overhangs, and connectivity to ensure feasible physical assembly. The results are demonstrated through the assembly of various objects, ranging from chairs to shelves, which are prompted via speech and realized within 5 minutes using a robotic arm.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 16 of its 44 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

16 of 44 references.

References · 16

  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 464
  3. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  4. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  5. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  6. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  7. 2023How Can Large Language Models Help Humans in Design and Manufacturing?cited by 41
  8. 2024LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesiscited by 39
  9. 2023An Automated Survey of Generative Artificial Intelligence: Large Language Models, Architectures, Protocols, and Applicationscited by 20
  10. 2023ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibilitycited by 17
  11. 2024Sketch2Prototype: Rapid Conceptual Design Exploration and Prototyping with Generative AIcited by 13
  12. 2023Style2Fab: Functionality-Aware Segmentation for Fabricating Personalized 3D Models with Generative AIcited by 11
4 more
  1. 2024Shaping Realities: Enhancing 3D Generative AI with Fabrication Constraintscited by 6
  2. 2024Toward Automated Programming for Robotic Assembly Using ChatGPTcited by 6
  3. 2024CADgpt: Harnessing Natural Language Processing for 3D Modelling to Enhance Computer-Aided Design Workflowscited by 4
  4. 2024AI in Manufacturing: Market Analysis and Opportunitiescited by 1

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Prompt-to-Product: Generative Assembly via Bimanual Manipulationcited by 1
  2. 2025Text to Robotic Assembly of Multi Component Objects using 3D Generative AI and Vision Language Modelscited by 1
  3. 2025AI Assisted AR Assembly: Object Recognition and Computer Vision for Augmented Reality Assisted Assemblycited by 1
  4. 2025Augmented Assembly: Object Recognition and Hand Tracking for Adaptive Assembly Instructions in Augmented Realitycited by 0