# PatentSBERTa: A Deep NLP based Hybrid Model for Patent Distance and Classification using Augmented SBERT

arXiv: https://arxiv.org/abs/2103.11933 · HTML: https://mlsanity.com/paper/2103.11933/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 55 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 7 of 7)

- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks](https://mlsanity.com/paper/2010.08240/) (2020)
- [ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms](https://mlsanity.com/paper/1807.05614/) (2018)
- [Patent Retrieval: A Literature Review](https://mlsanity.com/paper/1701.00324/) (2017)
- [TechNet: Technology Semantic Network Based on Patent Data](https://mlsanity.com/paper/1906.00411/) (2019)
- [A Text-Embedding-based Approach to Measure Patent-to-Patent Technological Similarity -- Workflow, Code, and Applications](https://mlsanity.com/paper/2003.12303/) (2020)
- [Mapping Technology Space by Normalizing Patent Networks](https://mlsanity.com/paper/1509.07285/) (2015)

## Built on by (most cited first, 24 of 24)

- [Natural Language Processing in the Patent Domain: A Survey](https://mlsanity.com/paper/2403.04105/) (2024)
- [A Survey on Patent Analysis: From NLP to Multimodal AI](https://mlsanity.com/paper/2404.08668/) (2024)
- [DAPFAM: A Domain-Aware Family-level Dataset to benchmark cross domain patent retrieval](https://mlsanity.com/paper/2506.22141/) (2025)
- [Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches](https://mlsanity.com/paper/2512.12677/) (2025)
- [Large Language Model for Patent Concept Generation](https://mlsanity.com/paper/2409.00092/) (2024)
- [Self-Filtered Distillation with LLMs-generated Trust Indicators for Reliable Patent Classification](https://mlsanity.com/paper/2510.05431/) (2025)
- [A Survey on Sentence Embedding Models Performance for Patent Analysis](https://mlsanity.com/paper/2206.02690/) (2022)
- [A comparative analysis of embedding models for patent similarity](https://mlsanity.com/paper/2403.16630/) (2024)
- [Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs](https://mlsanity.com/paper/2406.03614/) (2024)
- [AutoSpec: An Agentic Framework for Automatically Drafting Patent Specification](https://mlsanity.com/paper/2509.19640/) (2025)
- [Multi label classification of Artificial Intelligence related patents using Modified D2SBERT and Sentence Attention mechanism](https://mlsanity.com/paper/2303.03165/) (2023)
- [Unveiling Black-boxes: Explainable Deep Learning Models for Patent Classification](https://mlsanity.com/paper/2310.20478/) (2023)
- [Patent Language Model Pretraining with ModernBERT](https://mlsanity.com/paper/2509.14926/) (2025)
- [Patent Representation Learning via Self-supervision](https://mlsanity.com/paper/2511.10657/) (2025)
- [PINE: Pipeline for Important Node Exploration in Attributed Networks](https://mlsanity.com/paper/2512.07244/) (2025)
- [FullRecall: A Semantic Search-Based Ranking Approach for Maximizing Recall in Patent Retrieval](https://mlsanity.com/paper/2507.14946/) (2025)
- [From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models](https://mlsanity.com/paper/2509.09303/) (2025)
- [AI Patents in the United States and China: Measurement, Organization, and Knowledge Flows](https://mlsanity.com/paper/2604.10529/) (2026)
- [Anticipating Innovation Using Large Language Models](https://mlsanity.com/paper/2605.04875/) (2026)
- [Heterogeneous Dependency Graph-Guided Attentionfor Patent Representation Learning](https://mlsanity.com/paper/2605.10073/) (2026)
- [When Does Synthetic Patent Data Help? Volume-Fidelity Trade-offs in Low-Resource Multi-Label Classification](https://mlsanity.com/paper/2605.24296/) (2026)
- [Sparse Coverage: Semantic Center Representations for Patent Prior-Art Retrieval](https://mlsanity.com/paper/2608.16918/) (2026)
- [BLANC: Discovering Patent White Space via Changes in Normalized Pointwise Mutual Information Between Multi-View Clusters](https://mlsanity.com/paper/2608.26685/) (2026)
- [Benchmarking Embedding Geometries and Clustering Algorithms for Unsupervised Patent Claim Classification](https://mlsanity.com/paper/2610.12997/) (2026)

## Abstract (arXiv)

This study provides an efficient approach for using text data to calculate patent-to-patent (p2p) technological similarity, and presents a hybrid framework for leveraging the resulting p2p similarity for applications such as semantic search and automated patent classification. We create embeddings using Sentence-BERT (SBERT) based on patent claims. We leverage SBERTs efficiency in creating embedding distance measures to map p2p similarity in large sets of patent data. We deploy our framework for classification with a simple Nearest Neighbors (KNN) model that predicts Cooperative Patent Classification (CPC) of a patent based on the class assignment of the K patents with the highest p2p similarity. We thereby validate that the p2p similarity captures their technological features in terms of CPC overlap, and at the same demonstrate the usefulness of this approach for automatic patent classification based on text data. Furthermore, the presented classification framework is simple and the results easy to interpret and evaluate by end-users. In the out-of-sample model validation, we are able to perform a multi-label prediction of all assigned CPC classes on the subclass (663) level on 1,492,294 patents with an accuracy of 54% and F1 score > 66%, which suggests that our model outperforms the current state-of-the-art in text-based multi-label and multi-class patent classification. We furthermore discuss the applicability of the presented framework for semantic IP search, patent landscaping, and technology intelligence. We finally point towards a future research agenda for leveraging multi-source patent embeddings, their appropriateness across applications, as well as to improve and validate patent embeddings by creating domain-expert curated Semantic Textual Similarity (STS) benchmark datasets.