# IPBench: Benchmarking the Knowledge of Large Language Models in Intellectual Property

arXiv: https://arxiv.org/abs/2504.15524 · HTML: https://mlsanity.com/paper/2504.15524/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2025.

## Built on by (most cited first, 3 of 3)

- [When Reasoning Hurts Legal Drafting: The Verbalization Bottleneck in Patent Claim Generation](https://mlsanity.com/paper/2607.10480/) (2026)
- [PatRe: A Full-Stage Office Action and Rebuttal Generation Benchmark for Patent Examination](https://mlsanity.com/paper/2605.03571/) (2026)
- [Benchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering](https://mlsanity.com/paper/2605.24297/) (2026)

## Abstract (arXiv)

Intellectual Property (IP) is a highly specialized domain that integrates technical and legal knowledge, making it inherently complex and knowledge-intensive. Recent advancements in LLMs have demonstrated their potential to handle IP-related tasks, enabling more efficient analysis, understanding, and generation of IP-related content. However, existing datasets and benchmarks focus narrowly on patents or cover limited aspects of the IP field, lacking alignment with real-world scenarios. To bridge this gap, we introduce IPBench, the first comprehensive IP task taxonomy and a large-scale bilingual benchmark encompassing 8 IP mechanisms and 20 distinct tasks, designed to evaluate LLMs in real-world IP scenarios. We benchmark 17 main LLMs, ranging from general purpose to domain-specific, including chat-oriented and reasoning-focused models, under zero-shot, few-shot, and chain-of-thought settings. Our results show that even the top-performing model, DeepSeek-V3, achieves only 75.8% accuracy, indicating significant room for improvement. Notably, open-source IP and law-oriented models lag behind closed-source general-purpose models. To foster future research, we publicly release IPBench, and will expand it with additional tasks to better reflect real-world complexities and support model advancements in the IP domain. We provide the data and code in the supplementary URLs.