# The Effectiveness of Bidirectional Generative Patent Language Models

arXiv: https://arxiv.org/abs/2211.09690 · HTML: https://mlsanity.com/paper/2211.09690/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 13 references that are in the map.
- First posted on arXiv 4 Sep 2022.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Patent Claim Generation by Fine-Tuning OpenAI GPT-2](https://mlsanity.com/paper/1907.02052/) (2019)
- [Prior Art Search and Reranking for Generated Patent Text](https://mlsanity.com/paper/2009.09132/) (2020)
- [Evaluating Generative Patent Language Models](https://mlsanity.com/paper/2206.14578/) (2022)

## Built on by (most cited first, 2 of 2)

- [PatentVision: A multimodal method for drafting patent applications](https://mlsanity.com/paper/2510.09762/) (2025)
- [Patentformer: A demonstration of AI-assisted automated patent drafting](https://mlsanity.com/paper/2510.09752/) (2025)

## Abstract (arXiv)

Generative patent language models can assist humans to write patent text more effectively. The question is how to measure effectiveness from a human-centric perspective and how to improve effectiveness. In this manuscript, a simplified design of the autocomplete function is proposed to increase effectiveness by more than 10%. With the new design, the effectiveness of autocomplete can reach more than 60%, which means that more than 60% of keystrokes can be saved by autocomplete. Since writing patent text does not necessarily start from the beginning to the end, a question is whether the generative model can assist a user no matter where to start writing. To answer the question, the generative models in this manuscript are pre-trained with training data in both directions. The generative models become bidirectional. Since text generation is bidirectional, the calculation of autocomplete effectiveness can be bidirectional and starts from anywhere in the text. After thorough experiments, a key finding is that the autocomplete effectiveness of a model for the same text remains similar no matter where the calculation starts. The finding indicates that such bidirectional models can assist a user at a similar level, no matter where the user starts to write.