mlsanityData to 7 Oct 2026

Finding Generalizable Evidence by Learning to Convince Q&A Models

arXiv 1909.05863 ↗First posted 12 Sep 2019cs.CL cs.AI cs.IRcited by 25

Not on the map. Why?

Abstract (arXiv)

We propose a system that finds the strongest supporting evidence for a given answer to a question, using passage-based question-answering (QA) as a testbed. We train evidence agents to select the passage sentences that most convince a pretrained QA model of a given answer, if the QA model received those sentences instead of the full passage. Rather than finding evidence that convinces one model alone, we find that agents select evidence that generalizes; agent-chosen evidence increases the plausibility of the supported answer, as judged by other QA models and humans. Given its general nature, this approach improves QA in a robust manner: using agent-selected evidence (i) humans can correctly answer questions with only ~20% of the full passage and (ii) QA models can generalize to longer passages and harder questions.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2019: 1; 2020: 2; 2021: 5; 2022: 8; 2023: 4; 2024: 3; 2025: 2.

25 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2020Learning to summarize from human feedbackcited by 2,562
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2022Red Teaming Language Models with Language Modelscited by 983
  6. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  7. 2022Self-critiquing models for assisting human evaluatorscited by 372
  8. 2022Teaching language models to support answers with verified quotescited by 290
  9. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  10. 2021Explaining Answers with Entailment Treescited by 148
  11. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 147
  12. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
13 more
  1. 2022e-CARE: a New Dataset for Exploring Explainable Causal Reasoningcited by 47
  2. 2021Rissanen Data Analysis: Examining Dataset Characteristics via Description Lengthcited by 22
  3. 2023Sample Efficient Preference Alignment in LLMs via Active Explorationcited by 18
  4. 2021Robustifying Multi-hop QA through Pseudo-Evidentiality Trainingcited by 13
  5. 2023ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Taskcited by 12
  6. 2022JASMINE: Arabic GPT Models for Few-Shot Learningcited by 11
  7. 2021What's in your Head? Emergent Behaviour in Multi-Task Transformer Modelscited by 9
  8. 2025Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratingscited by 5
  9. 2023Kernelized Offline Contextual Dueling Banditscited by 4
  10. 2025Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labelingcited by 3
  11. 2024Anchored Alignment for Self-Explanations Enhancementcited by 1
  12. 2024Reward Design for Justifiable Sequential Decision-Makingcited by 0
  13. 2024Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awarenesscited by 0