# PEDANTIC: A Dataset for the Automatic Examination of Definiteness in Patent Claims

arXiv: https://arxiv.org/abs/2505.21342 · HTML: https://mlsanity.com/paper/2505.21342/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 14 of 14)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models](https://mlsanity.com/paper/2402.04614/) (2024)
- [Are self-explanations from Large Language Models faithful?](https://mlsanity.com/paper/2401.07927/) (2024)
- [Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations](https://mlsanity.com/paper/2310.11207/) (2023)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [Improving LLM-as-a-Judge Inference with the Judgment Distribution](https://mlsanity.com/paper/2503.03064/) (2025)
- [PatentMatch: A Dataset for Matching Patent Claims & Prior Art](https://mlsanity.com/paper/2012.13919/) (2020)
- [PatentEdits: Framing Patent Novelty as Textual Entailment](https://mlsanity.com/paper/2411.13477/) (2024)
- [Can AI Examine Novelty of Patents?: Novelty Evaluation Based on the Correspondence between Patent Claim and Prior Art](https://mlsanity.com/paper/2502.06316/) (2025)
- [ClaimCompare: A Data Pipeline for Evaluation of Novelty Destroying Patent Pairs](https://mlsanity.com/paper/2407.12193/) (2024)
- [Structural Representation Learning and Disentanglement for Evidential Chinese Patent Approval Prediction](https://mlsanity.com/paper/2408.12852/) (2024)
- [Comparing Complex Concepts with Transformers: Matching Patent Claims Against Natural Language Text](https://mlsanity.com/paper/2407.10351/) (2024)

## Built on by (most cited first, 3 of 3)

- [Is It Novel and Why? Fine-Grained Patent Novelty Prediction Based on Passage Retrieval](https://mlsanity.com/paper/2605.02392/) (2026)
- [PatRe: A Full-Stage Office Action and Rebuttal Generation Benchmark for Patent Examination](https://mlsanity.com/paper/2605.03571/) (2026)
- [Grounded Revision vs. Prior Injection: Probing Retrieval-Augmented Patent Claim Amendment](https://mlsanity.com/paper/2609.36550/) (2026)

## Abstract (arXiv)

Patent claims define the scope of protection for an invention. If there are ambiguities in a claim, it is rejected by the patent office. In the US, this is referred to as indefiniteness (35 U.S.C S 112(b)) and is among the most frequent reasons for patent application rejection. The development of automatic methods for patent definiteness examination has the potential to make patent drafting and examination more efficient, but no annotated dataset has been published to date. We introduce PEDANTIC (Patent Definiteness Examination Corpus), a novel dataset of 14k US patent claims from patent applications relating to Natural Language Processing (NLP), annotated with reasons for indefiniteness. We construct PEDANTIC using a fully automatic pipeline that retrieves office action documents from the USPTO and uses Large Language Models (LLMs) to extract the reasons for indefiniteness. A human validation study confirms the pipeline's accuracy in generating high-quality annotations. To gain insight beyond binary classification metrics, we implement an LLM-as-Judge evaluation that compares the free-form reasoning of every model-cited reason with every examiner-cited reason. We show that LLM agents based on Qwen 2.5 32B and 72B struggle to outperform logistic regression baselines on definiteness prediction, even though they often correctly identify the underlying reasons. PEDANTIC provides a valuable resource for patent AI researchers, enabling the development of advanced examination models. We will publicly release the dataset and code.