mlsanityData to 7 Oct 2026

To Answer or Not to Answer? Improving Machine Reading Comprehension Model with Span-based Contrastive Learning

arXiv 2208.01299 ↗First posted 2 Aug 2022cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Machine Reading Comprehension with Unanswerable Questions is a difficult NLP task, challenged by the questions which can not be answered from passages. It is observed that subtle literal changes often make an answerable question unanswerable, however, most MRC models fail to recognize such changes. To address this problem, in this paper, we propose a span-based method of Contrastive Learning (spanCL) which explicitly contrast answerable questions with their answerable and unanswerable counterparts at the answer span level. With spanCL, MRC models are forced to perceive crucial semantic changes from slight literal differences. Experiments on SQuAD 2.0 dataset show that spanCL can improve baselines significantly, yielding 0.86-2.14 absolute EM improvements. Additional experiments also show that spanCL is an effective way to utilize generated questions.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 50 of its 59 references that are in the map. First posted on arXiv 2 Aug 2022.

Builds on

50 of 59 references.

References · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  10. 2020Supervised Contrastive Learningcited by 2,803
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  12. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
38 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  2. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  3. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  4. 2015Teaching Machines to Read and Comprehendcited by 1,039
  5. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  6. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  7. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  8. 2018CoQA: A Conversational Question Answering Challengecited by 674
  9. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  10. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  11. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  12. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  13. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  14. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  15. 2019Contrastive Learning of Structured World Modelscited by 206
  16. 2019A Unified MRC Framework for Named Entity Recognitioncited by 199
  17. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  18. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  19. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  20. 2017Simple and Effective Multi-Paragraph Reading Comprehensioncited by 120
  21. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
  22. 2020Retrospective Reader for Machine Reading Comprehensioncited by 86
  23. 2019SG-Net: Syntax-Guided Machine Reading Comprehensioncited by 52
  24. 2021CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understandingcited by 50
  25. 2020Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Modelscited by 49
  26. 2021Paradigm Shift in Natural Language Processingcited by 39
  27. 2016Gated-Attention Readers for Text Comprehensioncited by 37
  28. 2017Making Neural QA as Simple as Possible but not Simplercited by 33
  29. 2020An Effective Transition-based Model for Discontinuous NERcited by 33
  30. 2016Attention-over-Attention Neural Networks for Reading Comprehensioncited by 31
  31. 2018Read + Verify: Machine Reading Comprehension with Unanswerable Questionscited by 31
  32. 2020Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Spacecited by 19
  33. 2016Text Understanding with the Attention Sum Reader Networkcited by 18
  34. 2020Conversational Machine Comprehension: a Literature Reviewcited by 10
  35. 2019Learning to Ask Unanswerable Questions for Machine Reading Comprehensioncited by 9
  36. 2020CAPT: Contrastive Pre-Training for Learning Denoised Sequence Representationscited by 8
  37. 2019Improving Machine Reading Comprehension via Adversarial Trainingcited by 7
  38. 2018Stochastic Answer Networks for SQuAD 2.0cited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2023A Lightweight Method to Generate Unanswerable Questions in Englishcited by 0
  2. 2026ViCLSR: A Supervised Contrastive Learning Framework with Natural Language Inference for Natural Language Understanding Taskscited by 0