Machine Reading Comprehension using Case-based Reasoning
Not on the map. Why?
Abstract (arXiv)
We present an accurate and interpretable method for answer extraction in machine reading comprehension that is reminiscent of case-based reasoning (CBR) from classical AI. Our method (CBR-MRC) builds upon the hypothesis that contextualized answers to similar questions share semantic similarities with each other. Given a test question, CBR-MRC first retrieves a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers in the retrieved cases. The semi-parametric nature of our approach allows it to attribute a prediction to the specific set of evidence cases, making it a desirable choice for building reliable and debuggable QA systems. We show that CBR-MRC provides high accuracy comparable with large reader models and outperforms baselines by 11.5 and 8.4 EM on NaturalQuestions and NewsQA, respectively. Further, we demonstrate the ability of CBR-MRC in identifying not just the correct answer tokens but also the span with the most relevant supporting evidence. Lastly, we observe that contexts for certain question types show higher lexical diversity than others and find that CBR-MRC is robust to these variations while performance using fully-parametric methods drops.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 57 of its 73 references that are in the map. First posted on arXiv 24 May 2023.
Builds on
57 of 73 references.
References · 57
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2020Supervised Contrastive Learningcited by 2,829
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
38 more
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
- 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
- 2015Teaching Machines to Read and Comprehendcited by 1,049
- 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
- 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
- 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
- 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
- 2017The NarrativeQA Reading Comprehension Challengecited by 597
- 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
- 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
- 2018QuAC : Question Answering in Contextcited by 381
- 2016NewsQA: A Machine Comprehension Datasetcited by 331
- 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
- 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
- 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 272
- 2022Binding Language Models in Symbolic Languagescited by 228
- 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 184
- 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 129
- 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
- 2019Analyzing and Improving Representations with the Soft Nearest Neighbor Losscited by 95
- 2019Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challengecited by 86
- 2018RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipescited by 74
- 2022On the Relation between Sensitivity and Accuracy in In-context Learningcited by 72
- 2019Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Indexcited by 64
- 2019Select, Answer and Explain: Interpretable Multi-hop Reading Comprehension over Multiple Documentscited by 57
- 2022Structured information extraction from complex scientific text with fine-tuned large language modelscited by 56
- 2020A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasetscited by 40
- 2022T-NER: An All-Round Python Library for Transformer-based Named Entity Recognitioncited by 37
- 2020BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QAcited by 36
- 2021Controllable Semantic Parsing via Retrieval Augmentationcited by 35
- 2020A Survey on Explainability in Machine Reading Comprehensioncited by 33
- 2022Knowledge Base Question Answering by Case-based Reasoning over Subgraphscited by 31
- 2016WikiReading: A Novel Large-scale Language Understanding Task over Wikipediacited by 26
- 2018Efficient and Robust Question Answering from Minimal Context over Documentscited by 26
- 2014Deep Learning for Answer Sentence Selectioncited by 17
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.