mlsanityData to 14 Oct 2026

Native Chinese Reader: A Dataset Towards Native-Level Chinese Machine Reading Comprehension

arXiv 2112.06494 ↗First posted 13 Dec 2021cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

We present Native Chinese Reader (NCR), a new machine reading comprehension (MRC) dataset with particularly long articles in both modern and classical Chinese. NCR is collected from the exam questions for the Chinese course in China's high schools, which are designed to evaluate the language proficiency of native Chinese youth. Existing Chinese MRC datasets are either domain-specific or focusing on short contexts of a few hundreds of characters in modern Chinese only. By contrast, NCR contains 8390 documents with an average length of 1024 characters covering a wide range of Chinese writing styles, including modern articles, classical literature and classical poetry. A total of 20477 questions on these documents also require strong reasoning abilities and common sense to figure out the correct answers. We implemented multiple baseline models using popular Chinese pre-trained models and additionally launched an online competition using our dataset to examine the limit of current methods. The best model achieves 59% test accuracy while human evaluation shows an average accuracy of 79%, which indicates a significant performance gap between current MRC models and native Chinese speakers. We release the dataset at https://sites.google.com/view/native-chinese-reader/.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 13 Dec 2021.

Builds on

35 of 53 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  9. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  10. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  11. 2015Teaching Machines to Read and Comprehendcited by 1,049
  12. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
23 more
  1. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  2. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  3. 2018CoQA: A Conversational Question Answering Challengecited by 683
  4. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  5. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  6. 2018QuAC : Question Answering in Contextcited by 381
  7. 2016NewsQA: A Machine Comprehension Datasetcited by 331
  8. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 302
  9. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 266
  10. 2019DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehensioncited by 156
  11. 2017DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applicationscited by 110
  12. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
  13. 2018A Span-Extraction Dataset for Chinese Machine Reading Comprehensioncited by 84
  14. 2019Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehensioncited by 81
  15. 2019ChID: A Large-scale Chinese IDiom Dataset for Cloze Testcited by 65
  16. 2018DRCD: a Chinese Machine Reading Comprehension Datasetcited by 57
  17. 2019SberQuAD -- Russian Reading Comprehension Dataset: Description and Analysiscited by 38
  18. 2017Large-scale Cloze Test Dataset Created by Teacherscited by 27
  19. 2016Who did What: A Large-Scale Person-Centered Cloze Datasetcited by 24
  20. 2016Embracing data abundance: BookTest Dataset for Reading Comprehensioncited by 15
  21. 2020A Sentence Cloze Dataset for Chinese Machine Reading Comprehensioncited by 14
  22. 2016Consensus Attention-based Neural Networks for Chinese Reading Comprehensioncited by 12
  23. 2017Dataset for the First Evaluation on Chinese Machine Reading Comprehensioncited by 9

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understandingcited by 19
  2. 2024LFED: A Literary Fiction Evaluation Dataset for Large Language Modelscited by 1