mlsanityData to 7 Oct 2026

Generative Transformers for Design Concept Generation

arXiv 2211.03468 ↗First posted 7 Nov 2022cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

Generating novel and useful concepts is essential during the early design stage to explore a large variety of design opportunities, which usually requires advanced design thinking ability and a wide range of knowledge from designers. Growing works on computer-aided tools have explored the retrieval of knowledge and heuristics from design data. However, they only provide stimuli to inspire designers from limited aspects. This study explores the recent advance of the natural language generation (NLG) technique in the artificial intelligence (AI) field to automate the early-stage design concept generation. Specifically, a novel approach utilizing the generative pre-trained transformer (GPT) is proposed to leverage the knowledge and reasoning from textual data and transform them into new concepts in understandable language. Three concept generation tasks are defined to leverage different knowledge and reasoning: domain knowledge synthesis, problem-driven synthesis, and analogy-driven synthesis. The experiments with both human and data-driven evaluation show good performance in generating novel and useful concepts.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 26 of its 122 references that are in the map. First posted on arXiv 7 Nov 2022.

Builds on

26 of 122 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  9. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
  10. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  11. 2021Deep Generative Models in Engineering Design: A Reviewcited by 103
  12. 2017Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responsescited by 98
14 more
  1. 2020Few-shot Natural Language Generation for Task-Oriented Dialogcited by 89
  2. 2020TopologyGAN: Topology Optimization Using Generative Adversarial Networks Based on Physical Fields Over the Initial Domaincited by 65
  3. 2022Putting GPT-3's Creativity to the (Alternative Uses) Testcited by 62
  4. 2017Paraphrase Generation with Deep Reinforcement Learningcited by 58
  5. 2019Patent Claim Generation by Fine-Tuning OpenAI GPT-2cited by 49
  6. 2020Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Modelscited by 27
  7. 2021Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNetcited by 27
  8. 2018Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generationcited by 21
  9. 2021Data-Driven Design-by-Analogy: State of the Art and Future Directionscited by 18
  10. 2021Engineering Knowledge Graph from Patent Databasecited by 18
  11. 2021Natural Language Processing in-and-for Design Researchcited by 15
  12. 2022Data-Driven Innovation: What Is Itcited by 10
  13. 2022Guiding Data-Driven Design Ideation by Knowledge Distancecited by 10
  14. 2022Design Representation as Semantic Networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 10; 2025: 3; 2026: 1.

21 papers.

Sort:

By area:

  1. 2024DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentationcited by 30
  2. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  3. 2023Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Designcited by 22
  4. 2024Large Language Model Agent as a Mechanical Designercited by 20
  5. 2023Conceptual Design Generation Using Large Language Modelscited by 13
  6. 2024AutoTRIZ: Automating Engineering Innovation with TRIZ and Large Language Modelscited by 8
  7. 2024Large Language Models for Combinatorial Optimization of Design Structure Matrixcited by 8
  8. 2023Toward Artificial Empathy for Human-Centered Design: A Frameworkcited by 6
  9. 2023Opportunities for Large Language Models and Discourse in Engineering Designcited by 6
  10. 2024Large Language Model for Patent Concept Generationcited by 4
  11. 2024A Novel Idea Generation Tool using a Structured Conversational AI (CAI) Systemcited by 4
  12. 2024Inspired by AI? A Novel Generative AI System To Assist Conceptual Automotive Designcited by 3
9 more
  1. 2023The Innovation Paradox: Concept Space Expansion with Diminishing Originality and the Promise of Creative AIcited by 2
  2. 2023Supermind Ideator: Exploring generative AI to support creative problem-solvingcited by 2
  3. 2024Exploring the Capabilities of Large Language Models for Generating Diverse Design Solutionscited by 2
  4. 2024Evaluating Large Language Models for Material Selectioncited by 2
  5. 2025Human-in-the-Loop: Quantitative Evaluation of 3D Models Generation by Large Language Modelscited by 2
  6. 2024MSEval: A Dataset for Material Selection in Conceptual Design to Evaluate Algorithmic Modelscited by 0
  7. 2025Large Language Models for Combinatorial Optimization of Design Structure Matrixcited by 0
  8. 2025Intelligent Design 4.0: Paradigm Evolution Toward the Agentic AI Eracited by 0
  9. 2026RePaint-Enhanced Conditional Diffusion Model for Parametric Engineering Designs under Performance and Parameter Constraintscited by 0