mlsanityData to 7 Oct 2026

PatentBERT: Patent Classification with Fine-Tuning a pre-trained BERT Model

arXiv 1906.02124 ↗First posted 14 May 2019cs.CL cs.LG stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

In this work we focus on fine-tuning a pre-trained BERT model and applying it to patent classification. When applied to large datasets of over two millions patents, our approach outperforms the state of the art by an approach using CNN with word embeddings. In addition, we focus on patent claims without other parts in patent documents. Our contributions include: (1) a new state-of-the-art method based on pre-trained BERT model and fine-tuning for patent classification, (2) a large dataset USPTO-3M at the CPC subclass level with SQL statements that can be used by future researchers, (3) showing that patent claims alone are sufficient for classification task, in contrast to conventional wisdom.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 5; 2023: 4; 2024: 6; 2025: 4.

23 papers.

Sort:

By area:

  1. 2024Semantic Similarity Matching for Patent Documents Using Ensemble BERT-related Model and Novel Text Processing Methodcited by 18
  2. 2023Towards General Purpose Medical AI: Continual Learning Medical Foundation Modelcited by 14
  3. 2021Biomedical and Clinical Language Models for Spanish: On the Benefits of Domain-Specific Pretraining in a Mid-Resource Scenariocited by 11
  4. 2021Linguistically Informed Masking for Representation Learning in the Patent Domaincited by 7
  5. 2024Reconciling Methodological Paradigms: Employing Large Language Models as Novice Qualitative Research Assistants in Talent Management Researchcited by 5
  6. 2025PatentMind: A Multi-Aspect Reasoning Graph for Patent Similarity Evaluationcited by 5
  7. 2023G2PTL: A Pre-trained Model for Delivery Address and its Applications in Logistics Systemcited by 4
  8. 2024Qualitative Insights Tool (QualIT): LLM Enhanced Topic Modelingcited by 4
  9. 2025Self-Filtered Distillation with LLMs-generated Trust Indicators for Reliable Patent Classificationcited by 4
  10. 2022Can Language Representation Models Think in Bets?cited by 3
  11. 2023A Novel Patent Similarity Measurement Methodology: Semantic Distance and Technological Distancecited by 3
  12. 2024A comparative analysis of embedding models for patent similaritycited by 3
11 more
  1. 2024PatentEval: Understanding Errors in Patent Generationcited by 3
  2. 2025Can AI Examine Novelty of Patents?: Novelty Evaluation Based on the Correspondence between Patent Claim and Prior Artcited by 3
  3. 2022Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probescited by 2
  4. 2024Beyond Scaling: Predicting Patent Approval with Domain-specific Fine-grained Claim Dependency Graphcited by 2
  5. 2021Solar cell patent classification method based on keyword extraction and deep neural networkcited by 1
  6. 2022K-12BERT: BERT for K-12 educationcited by 1
  7. 2023Large-Scale Text Analysis Using Generative Language Models: A Case Study in Discovering Public Value Expressions in AI Patentscited by 1
  8. 2021Introduction of a novel word embedding approach based on technology labels extracted from patent datacited by 0
  9. 2022Adaptation of domain-specific transformer models with text oversampling for sentiment analysis of social media posts on Covid-19 vaccinescited by 0
  10. 2022CinPatent: Datasets for Patent Classificationcited by 0
  11. 2025From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Modelscited by 0