# BERT based patent novelty search by training claims to their own description

arXiv: https://arxiv.org/abs/2103.01126 · HTML: https://mlsanity.com/paper/2103.01126/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [Prior Art Search and Reranking for Generated Patent Text](https://mlsanity.com/paper/2009.09132/) (2020)
- [Deep Patent Landscaping Model Using Transformer and Graph Embedding](https://mlsanity.com/paper/1903.05823/) (2019)

## Built on by (most cited first, 2 of 2)

- [BERT based freedom to operate patent analysis](https://mlsanity.com/paper/2105.00817/) (2021)
- [From Rules to Neural Graphs: Scalable Structured Prediction for Patent Prior Art Search](https://mlsanity.com/paper/2610.01553/) (2026)

## Abstract (arXiv)

In this paper we present a method to concatenate patent claims to their own description. By applying this method, BERT trains suitable descriptions for claims. Such a trained BERT (claim-to-description- BERT) could be able to identify novelty relevant descriptions for patents. In addition, we introduce a new scoring scheme, relevance scoring or novelty scoring, to process the output of BERT in a meaningful way. We tested the method on patent applications by training BERT on the first claims of patents and corresponding descriptions. BERT's output has been processed according to the relevance score and the results compared with the cited X documents in the search reports. The test showed that BERT has scored some of the cited X documents as highly relevant.