# CJRC: A Reliable Human-Annotated Benchmark DataSet for Chinese Judicial Reading Comprehension

arXiv: https://arxiv.org/abs/1912.09156 · HTML: https://mlsanity.com/paper/1912.09156/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2019.

## Built on by (most cited first, 26 of 26)

- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services](https://mlsanity.com/paper/2309.11325/) (2023)
- [Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents](https://mlsanity.com/paper/2105.03887/) (2021)
- [ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information](https://mlsanity.com/paper/2106.16038/) (2021)
- [A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Prediction](https://mlsanity.com/paper/2206.05224/) (2022)
- [A Statutory Article Retrieval Dataset in French](https://mlsanity.com/paper/2108.11792/) (2021)
- [Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2309.17050/) (2023)
- [LawGPT: A Chinese Legal Knowledge-Enhanced Large Language Model](https://mlsanity.com/paper/2406.04614/) (2024)
- [MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding](https://mlsanity.com/paper/2301.00876/) (2023)
- [VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models](https://mlsanity.com/paper/2305.12199/) (2023)
- [WYWEB: A NLP Evaluation Benchmark For Classical Chinese](https://mlsanity.com/paper/2305.14150/) (2023)
- [HLDC: Hindi Legal Documents Corpus](https://mlsanity.com/paper/2204.00806/) (2022)
- [Large Language Models Meet Legal Artificial Intelligence: A Survey](https://mlsanity.com/paper/2509.09969/) (2025)
- [LAiW: A Chinese Legal Large Language Models Benchmark](https://mlsanity.com/paper/2310.05620/) (2023)
- [Agent-Specific Deontic Modality Detection in Legal Language](https://mlsanity.com/paper/2211.12752/) (2022)
- [Event Grounded Criminal Court View Generation with Cooperative (Large) Language Models](https://mlsanity.com/paper/2404.07001/) (2024)
- [AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliance](https://mlsanity.com/paper/2510.01474/) (2025)
- [FedJudge: Federated Legal Large Language Model](https://mlsanity.com/paper/2309.08173/) (2023)
- [Optimizing Numerical Estimation and Operational Efficiency in the Legal Domain through Large Language Models](https://mlsanity.com/paper/2407.19041/) (2024)
- [VLQA: The First Comprehensive, Large, and High-Quality Vietnamese Dataset for Legal Question Answering](https://mlsanity.com/paper/2507.19995/) (2025)
- [CoAL-RAG: A Complexity-Aware Legal Retrieval-Augmented Generation Method](https://mlsanity.com/paper/2608.17536/) (2026)
- [Analyzing Hong Kong's Legal Judgments from a Computational Linguistics point-of-view](https://mlsanity.com/paper/2305.02558/) (2023)
- [LFED: A Literary Fiction Evaluation Dataset for Large Language Models](https://mlsanity.com/paper/2405.10166/) (2024)
- [Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2506.06060/) (2025)
- [CAPC-CG: A Large-Scale, Expert-Directed LLM-Annotated Corpus of Adaptive Policy Communication in China](https://mlsanity.com/paper/2510.08986/) (2025)
- [An Industrial-Scale Insurance LLM Achieving Verifiable Domain Mastery and Hallucination Control without Competence Trade-offs](https://mlsanity.com/paper/2603.14463/) (2026)

## Abstract (arXiv)

We present a Chinese judicial reading comprehension (CJRC) dataset which contains approximately 10K documents and almost 50K questions with answers. The documents come from judgment documents and the questions are annotated by law experts. The CJRC dataset can help researchers extract elements by reading comprehension technology. Element extraction is an important task in the legal field. However, it is difficult to predefine the element types completely due to the diversity of document types and causes of action. By contrast, machine reading comprehension technology can quickly extract elements by answering various questions from the long document. We build two strong baseline models based on BERT and BiDAF. The experimental results show that there is enough space for improvement compared to human annotators.