Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension
Not on the map. Why?
Abstract (arXiv)
Machine reading comprehension tasks require a machine reader to answer questions relevant to the given document. In this paper, we present the first free-form multiple-Choice Chinese machine reading Comprehension dataset (C^3), containing 13,369 documents (dialogues or more formally written mixed-genre texts) and their associated 19,577 multiple-choice free-form questions collected from Chinese-as-a-second-language examinations. We present a comprehensive analysis of the prior knowledge (i.e., linguistic, domain-specific, and general world knowledge) needed for these real-world problems. We implement rule-based and popular neural methods and find that there is still a significant performance gap between the best performing model (68.5%) and human readers (96.0%), especially on problems that require prior knowledge. We further study the effects of distractor plausibility and data augmentation based on translated relevant datasets for English on model performance. We expect C^3 to present great challenges to existing systems as answering 86.8% of questions requires both knowledge within and beyond the accompanying document, and we hope that C^3 can serve as a platform to study how to leverage various kinds of prior knowledge to better understand a given written or orally oriented text. C^3 is available at https://dataset.org/c3/.
Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Apr 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 11; 2022: 3; 2023: 15; 2024: 21; 2025: 17; 2026: 13.
80 papers.
Sort:
By area:
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
- 2023A Comprehensive Overview of Large Language Modelscited by 639
- 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
- 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
- 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
- 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
- 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
- 2024LaCo: Large Language Model Pruning via Layer Collapsecited by 124
- 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
- 2024Understanding Emergent Abilities of Language Models from the Loss Perspectivecited by 94
38 more
- 2023WebCPM: Interactive Web Search for Chinese Long-form Question Answeringcited by 93
- 2024Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencentcited by 89
- 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
- 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 78
- 2023DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuningcited by 64
- 2023PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computingcited by 55
- 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 55
- 2023Lawyer LLaMA Technical Reportcited by 54
- 2021CPM-2: Large-scale Cost-effective Pre-trained Language Modelscited by 52
- 2021ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 45
- 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharingcited by 38
- 2025Enhancing Multilingual LLM Pretraining with Model-Based Data Selectioncited by 33
- 2023Model Spider: Learning to Rank Pre-Trained Models Efficientlycited by 32
- 2024DDK: Distilling Domain Knowledge for Efficient Large Language Modelscited by 25
- 2024Pre-training Distillation for Large Language Models: A Design Space Explorationcited by 25
- 2025Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Modelscited by 22
- 2025A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedulescited by 22
- 2021TSQA: Tabular Scenario Based Question Answeringcited by 21
- 2023SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understandingcited by 18
- 2021CUGE: A Chinese Language Understanding and Generation Evaluation Benchmarkcited by 16
- 2025Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsitycited by 15
- 2023ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Humancited by 14
- 2024PanGu-π Pro:Rethinking Optimization and Architecture for Tiny Language Modelscited by 14
- 2024Tele-FLM Technical Reportcited by 12
- 2022Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learningcited by 11
- 2023Disco-Bench: A Discourse-Aware Evaluation Benchmark for Language Modellingcited by 10
- 2025ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearizationcited by 10
- 2024Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Modelscited by 9
- 2024OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safetycited by 9
- 2021Sub-Character Tokenization for Chinese Pretrained Language Modelscited by 7
- 2023KoRC: Knowledge oriented Reading Comprehension Benchmark for Deep Text Understandingcited by 7
- 2021Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Datacited by 6
- 2023ViMMRC 2.0 -- Enhancing Machine Reading Comprehension on Vietnamese Literature Textcited by 5
- 2024InternLM-Law: An Open Source Chinese Legal Large Language Modelcited by 5
- 2025dots.llm1 Technical Reportcited by 5
- 2021ExpMRC: Explainability Evaluation for Machine Reading Comprehensioncited by 4
- 2024YuLan: An Open-source Large Language Modelcited by 4
- 2025AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMscited by 4
50 most cited of 80 All 80 (JSON)
- 2026Chinese-Jev: Bringing System One Model to Chinese-Language Tasks29 Sep 2026
- 2026IronLLM: Forging Compact Edge-Native Language Models for Real-Time Embodied Intelligence29 Sep 2026
- 2026xHC: Expanded Hyper-Connections16 Jul 2026
- 2026The First ChineseBabyLM Challenge: training data-efficient and cognitively plausible language models for Chinese12 Jul 2026
- 2026Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models3 Jul 2026
- 2026CNM-BERT: A Drop-In Structural Embedding for Chinese Characters via Ideographic Description Sequences27 May 2026
- 2026NITP: Next Implicit Token Prediction for LLM Pre-training24 May 2026
- 2026SEED: Targeted Data Selection by Weighted Independent Set15 May 2026
- 2026UserGPT Technical Report9 May 2026
- 2026SimDiff: Depth Pruning via Similarity and Difference21 Apr 2026
- 2026GradMAP: Faster Layer Pruning with Gradient Metric and Projection Compensation16 Feb 2026
- 2026JTok: On Token Embedding as another Axis of Scaling Law via Joint Token Self-modulation31 Jan 2026
38 more
- 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models12 Jan 2026
- 2025Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimization22 Oct 2025
- 2025Investigating the Impact of Rationales for LLMs on Natural Language Understanding19 Oct 2025
- 2025Benchmarking Chinese Commonsense Reasoning with a Multi-hop Reasoning Perspective9 Oct 2025
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug 2025
- 2025Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective23 Aug 2025
- 2025Aligning Language Models with Real-time Knowledge Editing2 Aug 2025
- 2025AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMs24 Jul 2025
- 2025GPTailor: Large Language Model Pruning Through Layer Cutting and Stitching25 Jun 2025
- 2025dots.llm1 Technical Report6 Jun 2025
- 2025Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity27 May 2025
- 2025RLAP: A Reinforcement Learning Enhanced Adaptive Planning Framework for Multi-step NLP Task Solving17 May 2025
- 2025ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization5 May 2025
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models14 Apr 2025
- 2025A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules17 Mar 2025
- 2025Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Models17 Feb 2025
- 2025Enhancing Multilingual LLM Pretraining with Model-Based Data Selection14 Feb 2025
- 2025Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis6 Feb 2025
- 2024Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent4 Nov 2024
- 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing24 Oct 2024
- 2024Adaptive Segment-level Reward: Bridging the Gap Between Action and Reward Space in Alignment23 Oct 2024
- 2024Pre-training Distillation for Large Language Models: A Design Space Exploration21 Oct 2024
- 2024A Learning Rate Path Switching Training Paradigm for Version Updates of Large Language Models5 Oct 2024
- 2024RexUniNLU: Recursive Method with Explicit Schema Instructor for Universal NLU9 Sep 2024
- 2024DDK: Distilling Domain Knowledge for Efficient Large Language Models23 Jul 2024
- 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application2 Jul 2024
- 2024YuLan: An Open-source Large Language Model28 Jun 2024
- 2024InternLM-Law: An Open Source Chinese Legal Large Language Model21 Jun 2024
- 2024DGRC: An Effective Fine-tuning Framework for Distractor Generation in Chinese Multi-choice Reading Comprehension29 May 2024
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model7 May 2024
- 2024Tele-FLM Technical Report25 Apr 2024
- 2024Chinese Sequence Labeling with Semi-Supervised Boundary-Aware Language Model Pre-training8 Apr 2024
- 2024Understanding Emergent Abilities of Language Models from the Loss Perspective23 Mar 2024
- 2024OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safety18 Mar 2024
- 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expect6 Mar 2024
- 2024LaCo: Large Language Model Pruning via Layer Collapse17 Feb 2024
- 2024Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Models16 Feb 2024
- 2024PanGu-π Pro:Rethinking Optimization and Architecture for Tiny Language Models5 Feb 2024
50 newest of 80 All 80 (JSON)
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
- 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
- 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharingcited by 38
- 2024PanGu-π Pro:Rethinking Optimization and Architecture for Tiny Language Modelscited by 14
- 2025Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspectivecited by 1
- 2025Benchmarking Chinese Commonsense Reasoning with a Multi-hop Reasoning Perspectivecited by 1
- 2025Investigating the Impact of Rationales for LLMs on Natural Language Understandingcited by 0
- 2026GradMAP: Faster Layer Pruning with Gradient Metric and Projection Compensationcited by 0
- 2026GradMAP: Faster Layer Pruning with Gradient Metric and Projection Compensation16 Feb 2026
- 2025Investigating the Impact of Rationales for LLMs on Natural Language Understanding19 Oct 2025
- 2025Benchmarking Chinese Commonsense Reasoning with a Multi-hop Reasoning Perspective9 Oct 2025
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug 2025
- 2025Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective23 Aug 2025
- 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing24 Oct 2024
- 2024PanGu-π Pro:Rethinking Optimization and Architecture for Tiny Language Models5 Feb 2024
- 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermism5 Jan 2024
- 2023At Which Training Stage Does Code Data Help LLMs Reasoning?28 Sep 2023
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
- 2024Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencentcited by 89
- 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 78
- 2025Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsitycited by 15
- 2025dots.llm1 Technical Reportcited by 5
- 2026NITP: Next Implicit Token Prediction for LLM Pre-trainingcited by 3
- 2026xHC: Expanded Hyper-Connectionscited by 3
- 2026JTok: On Token Embedding as another Axis of Scaling Law via Joint Token Self-modulationcited by 2
- 2026xHC: Expanded Hyper-Connections16 Jul 2026
- 2026NITP: Next Implicit Token Prediction for LLM Pre-training24 May 2026
- 2026JTok: On Token Embedding as another Axis of Scaling Law via Joint Token Self-modulation31 Jan 2026
- 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models12 Jan 2026
- 2025dots.llm1 Technical Report6 Jun 2025
- 2025Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity27 May 2025
- 2024Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent4 Nov 2024
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model7 May 2024
- 2025Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Modelscited by 22
- 2024Adaptive Segment-level Reward: Bridging the Gap Between Action and Reward Space in Alignmentcited by 2
- 2025Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesiscited by 1
- 2025Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimizationcited by 1
- 2025Aligning Language Models with Real-time Knowledge Editingcited by 0
- 2026Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Modelscited by 0
- 2026Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models3 Jul 2026
- 2025Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimization22 Oct 2025
- 2025Aligning Language Models with Real-time Knowledge Editing2 Aug 2025
- 2025Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Models17 Feb 2025
- 2025Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis6 Feb 2025
- 2024Adaptive Segment-level Reward: Bridging the Gap Between Action and Reward Space in Alignment23 Oct 2024
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2023A Comprehensive Overview of Large Language Modelscited by 639
- 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
- 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
- 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
- 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
- 2024LaCo: Large Language Model Pruning via Layer Collapsecited by 124
- 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
- 2024Understanding Emergent Abilities of Language Models from the Loss Perspectivecited by 94
- 2023WebCPM: Interactive Web Search for Chinese Long-form Question Answeringcited by 93
- 2023PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computingcited by 55
- 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 55
12 most cited of 55 All 55 (JSON)
- 2026Chinese-Jev: Bringing System One Model to Chinese-Language Tasks29 Sep 2026
- 2026IronLLM: Forging Compact Edge-Native Language Models for Real-Time Embodied Intelligence29 Sep 2026
- 2026The First ChineseBabyLM Challenge: training data-efficient and cognitively plausible language models for Chinese12 Jul 2026
- 2026CNM-BERT: A Drop-In Structural Embedding for Chinese Characters via Ideographic Description Sequences27 May 2026
- 2026SEED: Targeted Data Selection by Weighted Independent Set15 May 2026
- 2026UserGPT Technical Report9 May 2026
- 2026SimDiff: Depth Pruning via Similarity and Difference21 Apr 2026
- 2025GPTailor: Large Language Model Pruning Through Layer Cutting and Stitching25 Jun 2025
- 2025RLAP: A Reinforcement Learning Enhanced Adaptive Planning Framework for Multi-step NLP Task Solving17 May 2025
- 2025ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization5 May 2025
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models14 Apr 2025
- 2025A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules17 Mar 2025
12 newest of 55 All 55 (JSON)