mlsanityData to 14 Oct 2026

IDK-MRC: Unanswerable Questions for Indonesian Machine Reading Comprehension

arXiv 2210.13778 ↗First posted 25 Oct 2022cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Machine Reading Comprehension (MRC) has become one of the essential tasks in Natural Language Understanding (NLU) as it is often included in several NLU benchmarks (Liang et al., 2020; Wilie et al., 2020). However, most MRC datasets only have answerable question type, overlooking the importance of unanswerable questions. MRC models trained only on answerable questions will select the span that is most likely to be the answer, even when the answer does not actually exist in the given passage (Rajpurkar et al., 2018). This problem especially remains in medium- to low-resource languages like Indonesian. Existing Indonesian MRC datasets (Purwarianti et al., 2007; Clark et al., 2020) are still inadequate because of the small size and limited question types, i.e., they only cover answerable questions. To fill this gap, we build a new Indonesian MRC dataset called I(n)don'tKnow- MRC (IDK-MRC) by combining the automatic and manual unanswerable question generation to minimize the cost of manual dataset construction while maintaining the dataset quality. Combined with the existing answerable questions, IDK-MRC consists of more than 10K questions in total. Our analysis shows that our dataset significantly improves the performance of Indonesian MRC models, showing a large improvement for unanswerable questions.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

31 of 38 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  8. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  9. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  10. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  11. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  12. 2016NewsQA: A Machine Comprehension Datasetcited by 331
19 more
  1. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
  2. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  3. 2017Learning to Ask: Neural Question Generation for Reading Comprehensioncited by 186
  4. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  5. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  6. 2020IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understandingcited by 109
  7. 2020Training Question Answering Models From Synthetic Datacited by 95
  8. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 79
  9. 2020End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systemscited by 65
  10. 2019Cross-Lingual Natural Language Generation via Pre-Trainingcited by 63
  11. 2019Unsupervised Question Answering by Cloze Translationcited by 59
  12. 2018Harvesting Paragraph-Level Question-Answer Pairs from Wikipediacited by 49
  13. 2020What do Models Learn from Question Answering Datasets?cited by 45
  14. 2020Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answeringcited by 41
  15. 2019Cross-Lingual Training for Automatic Question Generationcited by 23
  16. 2019Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worldscited by 19
  17. 2020Towards Zero-Shot Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehensioncited by 17
  18. 2019Learning to Ask Unanswerable Questions for Machine Reading Comprehensioncited by 9
  19. 2020Sequence-to-Sequence Learning for Indonesian Automatic Question Generatorcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 1.

4 papers.

  1. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
  2. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  3. 2023QASiNa: Religious Domain Question Answering using Sirah Nabawiyahcited by 15
  4. 2024TIGQA:An Expert Annotated Question Answering Dataset in Tigrinyacited by 5