# Large Language Model for Patent Concept Generation

arXiv: https://arxiv.org/abs/2409.00092 · HTML: https://mlsanity.com/paper/2409.00092/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 16 of 16)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [HuatuoGPT, towards Taming Language Model to Be a Doctor](https://mlsanity.com/paper/2305.15075/) (2023)
- [Patent Claim Generation by Fine-Tuning OpenAI GPT-2](https://mlsanity.com/paper/1907.02052/) (2019)
- [PatentSBERTa: A Deep NLP based Hybrid Model for Patent Distance and Classification using Augmented SBERT](https://mlsanity.com/paper/2103.11933/) (2021)
- [Generative Transformers for Design Concept Generation](https://mlsanity.com/paper/2211.03468/) (2022)
- [Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity](https://mlsanity.com/paper/2401.07348/) (2024)
- [Generative Pre-Trained Transformer for Design Concept Generation: An Exploration](https://mlsanity.com/paper/2111.08489/) (2021)
- [Data-Driven Innovation: What Is It](https://mlsanity.com/paper/2201.08184/) (2022)
- [MoZIP: A Multilingual Benchmark to Evaluate Large Language Models in Intellectual Property](https://mlsanity.com/paper/2402.16389/) (2024)
- [Evaluating Generative Patent Language Models](https://mlsanity.com/paper/2206.14578/) (2022)
- [Generative Design Ideation: A Natural Language Generation Approach](https://mlsanity.com/paper/2204.09658/) (2022)
- [From PARIS to LE-PARIS: Toward Patent Response Automation with Recommender Systems and Collaborative Large Language Models](https://mlsanity.com/paper/2402.00421/) (2024)
- [The Innovation Paradox: Concept Space Expansion with Diminishing Originality and the Promise of Creative AI](https://mlsanity.com/paper/2303.13300/) (2023)

## Built on by (most cited first, 4 of 4)

- [Pointer-Augmented Autoregressive Generation of Patent Claims with Joint Topology and Content Decoding](https://mlsanity.com/paper/2607.24040/) (2026)
- [Intelligent Design 4.0: Paradigm Evolution Toward the Agentic AI Era](https://mlsanity.com/paper/2506.09755/) (2025)
- [Self-Knowledge Retrieval Augmented Generation Framework for Patent Matching](https://mlsanity.com/paper/2608.11030/) (2026)
- [LogicTree-RAG: Logic Tree-guided Retrieval-Augmented Generation for Long-form Patent Drafting](https://mlsanity.com/paper/2609.30943/) (2026)

## Abstract (arXiv)

In traditional innovation practices, concept and IP generation are often iteratively integrated. Both processes demand an intricate understanding of advanced technical domain knowledge. Existing large language models (LLMs), while possessing massive pre-trained knowledge, often fall short in the innovative concept generation due to a lack of specialized knowledge necessary for the generation. To bridge this critical gap, we propose a novel knowledge finetuning (KFT) framework to endow LLM-based AI with the ability to autonomously mine, understand, and apply domain-specific knowledge and concepts for invention generation, i.e., concept and patent generation together. Our proposed PatentGPT integrates knowledge injection pre-training (KPT), domain-specific supervised finetuning (SFT), and reinforcement learning from human feedback (RLHF). Extensive evaluation shows that PatentGPT significantly outperforms the state-of-the-art models on patent-related benchmark tests. Our method not only provides new insights into data-driven innovation but also paves a new path to fine-tune LLMs for applications in the context of technology. We also discuss the managerial and policy implications of AI-generating inventions in the future.