# Generative Pre-Trained Transformers for Biologically Inspired Design

arXiv: https://arxiv.org/abs/2204.09714 · HTML: https://mlsanity.com/paper/2204.09714/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 49 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [Re-evaluating Automatic Metrics for Image Captioning](https://mlsanity.com/paper/1612.07600/) (2016)
- [Data-Driven Design-by-Analogy: State of the Art and Future Directions](https://mlsanity.com/paper/2106.01592/) (2021)
- [Generative Pre-Trained Transformer for Design Concept Generation: An Exploration](https://mlsanity.com/paper/2111.08489/) (2021)
- [Natural Language Processing in-and-for Design Research](https://mlsanity.com/paper/2111.13827/) (2021)

## Built on by (most cited first, 1 of 1)

- [The Innovation Paradox: Concept Space Expansion with Diminishing Originality and the Promise of Creative AI](https://mlsanity.com/paper/2303.13300/) (2023)

## Abstract (arXiv)

Biological systems in nature have evolved for millions of years to adapt and survive the environment. Many features they developed can be inspirational and beneficial for solving technical problems in modern industries. This leads to a novel form of design-by-analogy called bio-inspired design (BID). Although BID as a design method has been proven beneficial, the gap between biology and engineering continuously hinders designers from effectively applying the method. Therefore, we explore the recent advance of artificial intelligence (AI) for a computational approach to bridge the gap. This paper proposes a generative design approach based on the pre-trained language model (PLM) to automatically retrieve and map biological analogy and generate BID in the form of natural language. The latest generative pre-trained transformer, namely GPT-3, is used as the base PLM. Three types of design concept generators are identified and fine-tuned from the PLM according to the looseness of the problem space representation. Machine evaluators are also fine-tuned to assess the correlation between the domains within the generated BID concepts. The approach is then tested via a case study in which the fine-tuned models are applied to generate and evaluate light-weighted flying car concepts inspired by nature. The results show our approach can generate BID concepts with good performance.