# PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims

arXiv: https://arxiv.org/abs/2505.19345 · HTML: https://mlsanity.com/paper/2505.19345/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 27 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Large Language Models in Law: A Survey](https://mlsanity.com/paper/2312.03718/) (2023)
- [Patent Claim Generation by Fine-Tuning OpenAI GPT-2](https://mlsanity.com/paper/1907.02052/) (2019)
- [The Harvard USPTO Patent Dataset: A Large-Scale, Well-Structured, and Multi-Purpose Corpus of Patent Applications](https://mlsanity.com/paper/2207.04043/) (2022)

## Built on by (most cited first, 11 of 11)

- [Pat-DEVAL: Chain-of-Legal-Thought Evaluation for Patent Description](https://mlsanity.com/paper/2601.00166/) (2026)
- [FlowPlan-G2P: A Structured Generation Framework for Transforming Scientific Papers into Patent Descriptions](https://mlsanity.com/paper/2601.02589/) (2026)
- [Adaptive Cost-Efficient Evaluation for Reliable Patent Claim Generation](https://mlsanity.com/paper/2604.04295/) (2026)
- [Formally Verified Patent Analysis via Dependent Type Theory: Machine-Checkable Certificates from a Hybrid AI + Lean 4 Pipeline](https://mlsanity.com/paper/2604.18882/) (2026)
- [Pointer-Augmented Autoregressive Generation of Patent Claims with Joint Topology and Content Decoding](https://mlsanity.com/paper/2607.24040/) (2026)
- [Can Large Language Models Understand As Well As Apply Patent Regulations to Pass a Hands-On Patent Attorney Test?](https://mlsanity.com/paper/2507.10576/) (2025)
- [ERA-IT: Aligning Semantic Models with Revealed Economic Preference for Real-Time and Explainable Patent Valuation](https://mlsanity.com/paper/2512.12869/) (2025)
- [Benchmarking Patent Drafting from Inventor-Style Disclosures](https://mlsanity.com/paper/2608.21249/) (2026)
- [Vibe Patenting: Evaluating LLM Judges for Professional Patent-Drafting Agents](https://mlsanity.com/paper/2609.13422/) (2026)
- [LogicTree-RAG: Logic Tree-guided Retrieval-Augmented Generation for Long-form Patent Drafting](https://mlsanity.com/paper/2609.30943/) (2026)
- [Grounded Revision vs. Prior Injection: Probing Retrieval-Augmented Patent Claim Amendment](https://mlsanity.com/paper/2609.36550/) (2026)

## Abstract (arXiv)

High-stakes texts such as patent claims, medical records, and technical reports are structurally complex and demand a high degree of reliability and precision. While large language models (LLMs) have recently been applied to automate their generation in high-stakes domains, reliably evaluating such outputs remains a major challenge. Conventional natural language generation (NLG) metrics are effective for generic documents but fail to capture the structural and legal characteristics essential to evaluating complex high-stakes documents. To address this gap, we propose PatentScore, a multi-dimensional evaluation framework specifically designed for one of the most intricate and rigorous domains, patent claims. PatentScore integrates hierarchical decomposition of claim elements, validation patterns grounded in legal and technical standards, and scoring across structural, semantic, and legal dimensions. In experiments on our dataset which consists of 400 Claim1, PatentScore achieved the highest correlation with expert annotations (r = 0.819), significantly outperforming widely used NLG metrics. This work establishes a new standard for evaluating LLM-generated patent claims, providing a solid foundation for research on patent generation and validation.