mlsanityData to 7 Oct 2026

Learning with Instance Bundles for Reading Comprehension

arXiv 2104.08735 ↗First posted 18 Apr 2021cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other. However, closely related questions and their corresponding answers are not independent, and leveraging these relationships could provide a strong supervision signal to a model. Drawing on ideas from contrastive estimation, we introduce several new supervision techniques that compare question-answer scores across multiple related instances. Specifically, we normalize these scores across various neighborhoods of closely contrasting questions and/or answers, adding another cross entropy loss term that is used in addition to traditional maximum likelihood estimation. Our techniques require bundles of related question-answer pairs, which we can either mine from within existing data or create using various automated heuristics. We empirically demonstrate the effectiveness of training with instance bundles on two datasets -- HotpotQA and ROPES -- showing up to 11% absolute gains in accuracy.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

31 of 44 references.

References · 31

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  4. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  5. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  6. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  7. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  8. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  9. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  10. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  11. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  12. 2019Good-Enough Compositional Data Augmentationcited by 158
19 more
  1. 2020Explaining NLP Models via Minimal Contrastive Editing (MiCE)cited by 105
  2. 2019Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoningcited by 100
  3. 2019Reasoning Over Paragraph Effects in Situationscited by 79
  4. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 77
  5. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  6. 2020Learning What Makes a Difference from Counterfactual Examples and Gradient Supervisioncited by 66
  7. 2019A Logic-Driven Framework for Consistency of Neural Modelscited by 64
  8. 2020Explaining The Efficacy of Counterfactually Augmented Datacited by 64
  9. 2017Contrastive Learning for Image Captioningcited by 54
  10. 2018Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledgecited by 45
  11. 2020Group-wise Contrastive Learning for Neural Dialogue Generationcited by 15
  12. 2019An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Searchcited by 12
  13. 2019QAInfomax: Learning Robust Question Answering System by Mutual Information Maximizationcited by 9
  14. 2020RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answeringcited by 9
  15. 2020Multi-Step Inference for Reasoning Over Paragraphscited by 8
  16. 2020Cascaded Text Generation with Markov Transformerscited by 7
  17. 2020An Analysis of the Utility of Explicit Negative Examples to Improve the Syntactic Abilities of Neural Language Modelscited by 6
  18. 2021Paired Examples as Indirect Supervision in Latent Decision Modelscited by 6
  19. 2019On Incorporating Semantic Prior Knowledge in Deep Learning Through Embedding-Space Constraintscited by 4

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 2; 2024: 1.

8 papers.

  1. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  2. 2021Retrieval-guided Counterfactual Generation for QAcited by 28
  3. 2022CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generationcited by 23
  4. 2021Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decompositioncited by 11
  5. 2022Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformationcited by 5
  6. 2023Hallucination Augmented Recitations for Language Modelscited by 5
  7. 2023How Much Consistency Is Your Accuracy Worth?cited by 1
  8. 2024Improving QA Model Performance with Cartographic Inoculationcited by 1