mlsanityData to 14 Oct 2026

Generative Pre-Trained Transformer for Design Concept Generation: An Exploration

arXiv 2111.08489 ↗First posted 16 Nov 2021cs.CL cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Novel concepts are essential for design innovation and can be generated with the aid of data stimuli and computers. However, current generative design algorithms focus on diagrammatic or spatial concepts that are either too abstract to understand or too detailed for early phase design exploration. This paper explores the uses of generative pre-trained transformers (GPT) for natural language design concept generation. Our experiments involve the use of GPT-2 and GPT-3 for different creative reasonings in design tasks. Both show reasonably good performance for verbal design concept generation.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 11 of its 61 references that are in the map. First posted on arXiv 16 Nov 2021.

Builds on

11 of 61 references.

References · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  5. 2021Deep Generative Models in Engineering Design: A Reviewcited by 105
  6. 2020Few-shot Natural Language Generation for Task-Oriented Dialogcited by 89
  7. 2020TopologyGAN: Topology Optimization Using Generative Adversarial Networks Based on Physical Fields Over the Initial Domaincited by 65
  8. 2017Paraphrase Generation with Deep Reinforcement Learningcited by 58
  9. 2019Patent Claim Generation by Fine-Tuning OpenAI GPT-2cited by 49
  10. 2021Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNetcited by 27
  11. 2018Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generationcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 7; 2024: 3; 2025: 3; 2026: 1.

17 papers.

Sort:

By area:

  1. 2023Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directionscited by 55
  2. 2023Fluid Transformers and Creative Analogies: Exploring Large Language Models' Capacity for Augmenting Cross-Domain Analogical Creativitycited by 23
  3. 2021Natural Language Processing in-and-for Design Researchcited by 15
  4. 2022Biologically Inspired Design Concept Generation Using Generative Pre-Trained Transformerscited by 13
  5. 2023Conceptual Design Generation Using Large Language Modelscited by 13
  6. 2023Mapping the Design Space of Interactions in Human-AI Text Co-creation Taskscited by 6
  7. 2023Opportunities for Large Language Models and Discourse in Engineering Designcited by 6
  8. 2024Reviewing Intelligent Cinematography: AI research for camera-based video productioncited by 4
  9. 2024Large Language Model for Patent Concept Generationcited by 4
  10. 2023Elucidating STEM Concepts through Generative AI: A Multi-modal Exploration of Analogical Reasoningcited by 3
  11. 2023The Innovation Paradox: Concept Space Expansion with Diminishing Originality and the Promise of Creative AIcited by 2
  12. 2024ChatGPT Alternative Solutions: Large Language Models Surveycited by 2
5 more
  1. 2025Human-in-the-Loop: Quantitative Evaluation of 3D Models Generation by Large Language Modelscited by 2
  2. 2022Generative Pre-Trained Transformers for Biologically Inspired Designcited by 1
  3. 2025LLM-Based Design Pattern Detectioncited by 0
  4. 2025RECALL-MM: A Multimodal Dataset of Consumer Product Recalls for Risk Analysis using Computational Methods and Large Language Modelscited by 0
  5. 2026Design-OS: A Specification-Driven Framework for Engineering System Design with a Control-Systems Design Casecited by 0