Enabling Generative Design Tools with LLM Agents for Mechanical Computation Devices: A Case Study
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
In the field of Human-Computer Interaction (HCI), interactive devices with embedded mechanical computation are gaining attention. The rise of these cutting-edge devices has created a need for specialized design tools that democratize the prototyping process. While current tools streamline prototyping through parametric design and simulation, they often come with a steep learning curve and may not fully support creative ideation. In this study, we use fluidic computation interfaces as a case study to explore how design tools for such devices can be augmented by Large Language Model agents (LLMs). Integrated with LLMs, the Generative Design Tool (GDT) better understands the capabilities and limitations of new technologies, proposes diverse and practical applications, and suggests designs that are technically and contextually appropriate. Additionally, it generates design parameters for visualizing results and producing fabrication-ready support files. This paper details the GDT's framework, implementation, and performance while addressing its potential and challenges.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 84 references that are in the map. First posted on arXiv 28 May 2024.
Builds on
26 of 84 references.
References · 26
- 2017Attention Is All You Needcited by 59,932
- 2023GPT-4 Technical Reportcited by 18,132
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
- 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
- 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
14 more
- 2023Using an LLM to Help With Code Understandingcited by 222
- 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
- 2023Chip-Chat: Challenges and Opportunities in Conversational Hardware Designcited by 136
- 2023BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questionscited by 125
- 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
- 2021JoinABLe: Learning Bottom-up Assembly of Parametric CAD Jointscited by 77
- 2023Graphologue: Exploring Large Language Model Responses with Interactive Diagramscited by 65
- 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaborationcited by 45
- 2023RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressionscited by 43
- 20223DALL-E: Integrating Text-to-Image AI in 3D Design Workflowscited by 41
- 2023Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Promptingcited by 36
- 2023Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chainscited by 24
- 2023Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Toolscited by 16
- 2020SimuLearn: Fast and Accurate Simulator to Support Morphing Materials Design and Workflowscited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Conversational Interfaces for Parametric Conceptual Architectural Design: Integrating Mixed Reality with LLM-driven Interaction
- 2026LLM for EDA in Front-End Design: Challenges and Opportunities
- 2025AI Agents in Engineering Design: A Multi-Agent Framework for Aesthetic and Aerodynamic Car Design
- 2024Towards a copilot in BIM authoring tool using a large language model-based agent for intelligent human-machine interaction
- 2026LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures
- 2025A Multidisciplinary Design and Optimization (MDO) Agent Driven by Large Language Models
- 2024Mediating Modes of Thought: LLM's for design scripting
- 2025GraphicBench: A Planning Benchmark for Graphic Design with Language Agents