# InstructPatentGPT: Training patent language models to follow instructions with human feedback

arXiv: https://arxiv.org/abs/2406.16897 · HTML: https://mlsanity.com/paper/2406.16897/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 25 May 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 14 of 14)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022)
- [8-bit Optimizers via Block-wise Quantization](https://mlsanity.com/paper/2110.02861/) (2021)
- [Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization](https://mlsanity.com/paper/2210.01241/) (2022)
- [Offline RL for Natural Language Generation with Implicit Language Q Learning](https://mlsanity.com/paper/2206.11871/) (2022)
- [Patent Claim Generation by Fine-Tuning OpenAI GPT-2](https://mlsanity.com/paper/1907.02052/) (2019)
- [Evaluating Generative Patent Language Models](https://mlsanity.com/paper/2206.14578/) (2022)
- [Automated patent extraction powers generative modeling in focused chemical spaces](https://mlsanity.com/paper/2303.08272/) (2023)
- [Large-Scale Text Analysis Using Generative Language Models: A Case Study in Discovering Public Value Expressions in AI Patents](https://mlsanity.com/paper/2305.10383/) (2023)

## Built on by (most cited first, 19 of 19)

- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Pap2Pat: Benchmarking Outline-Guided Long-Text Patent Generation with Patent-Paper Pairs](https://mlsanity.com/paper/2410.07009/) (2024)
- [Where Hindsight Credit Can Reside: A Signed-Capacity View of Token Updates in RLVR](https://mlsanity.com/paper/2604.11056/) (2026)
- [Compute-Optimal Quantization-Aware Training](https://mlsanity.com/paper/2509.22935/) (2025)
- [LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble](https://mlsanity.com/paper/2411.17135/) (2024)
- [Harnessing RLHF for Robust Unanswerability Recognition and Trustworthy Response Generation in LLMs](https://mlsanity.com/paper/2507.16951/) (2025)
- [VULPO: Context-Aware Vulnerability Detection via On-Policy LLM Optimization](https://mlsanity.com/paper/2511.11896/) (2025)
- [Do LLMs Know They Are Being Tested? Evaluation Awareness and Incentive-Sensitive Failures in GPT-OSS-20B](https://mlsanity.com/paper/2510.08624/) (2025)
- [Enhancing Contrastive Demonstration Selection with Semantic Diversity for Robust In-Context Machine Translation](https://mlsanity.com/paper/2504.09305/) (2025)
- [Contextual Candor: Enhancing LLM Trustworthiness Through Hierarchical Unanswerability Detection](https://mlsanity.com/paper/2506.01104/) (2025)
- [TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection](https://mlsanity.com/paper/2507.03673/) (2025)
- [Can Large Language Models Understand As Well As Apply Patent Regulations to Pass a Hands-On Patent Attorney Test?](https://mlsanity.com/paper/2507.10576/) (2025)
- [ERA-IT: Aligning Semantic Models with Revealed Economic Preference for Real-Time and Explainable Patent Valuation](https://mlsanity.com/paper/2512.12869/) (2025)
- [On Temperature-Constrained Non-Deterministic Machine Translation: Potential and Evaluation](https://mlsanity.com/paper/2601.13729/) (2026)
- [AR-MAP: Are Autoregressive Large Language Models Implicit Teachers for Diffusion Large Language Models?](https://mlsanity.com/paper/2602.02178/) (2026)
- [Clouding the Mirror: Stealthy Prompt Injection Attacks Targeting LLM-based Phishing Detection](https://mlsanity.com/paper/2602.05484/) (2026)
- [Inventorship in AI-Assisted Inventions: Designing an Experiment to Shape Case Law](https://mlsanity.com/paper/2605.16528/) (2026)
- [LogicTree-RAG: Logic Tree-guided Retrieval-Augmented Generation for Long-form Patent Drafting](https://mlsanity.com/paper/2609.30943/) (2026)
- [Grounded Revision vs. Prior Injection: Probing Retrieval-Augmented Patent Claim Amendment](https://mlsanity.com/paper/2609.36550/) (2026)

## Abstract (arXiv)

In this research, patent prosecution is conceptualized as a system of reinforcement learning from human feedback. The objective of the system is to increase the likelihood for a language model to generate patent claims that have a higher chance of being granted. To showcase the controllability of the language model, the system learns from granted patents and pre-grant applications with different rewards. The status of "granted" and "pre-grant" are perceived as labeled human feedback implicitly. In addition, specific to patent drafting, the experiments in this research demonstrate the model's capability to learn from adjusting claim length and inclusion of limiting terms for narrowing claim scope. As proof of concept, the experiments focus on claim ones only and the training data originates from a patent dataset tailored specifically for artificial intelligence. Although the available human feedback in patent prosecution are limited and the quality of generated patent text requires improvement, the experiments following the 3-stage reinforcement learning from human feedback have demonstrated that generative language models are capable of reflecting the human feedback or intent in patent prosecution. To enhance the usability of language models, the implementation in this research utilizes modern techniques that enable execution on a single consumer-grade GPU. The demonstrated proof of concept, which reduces hardware requirements, will prove valuable in the future as more human feedback in patent prosecution become available for broader use, either within patent offices or in the public domain.