mlsanityData to 14 Oct 2026

Improving Factual Consistency of Abstractive Summarization via Question Answering

arXiv 2105.04623 ↗First posted 10 May 2021cs.CL cs.AIcited by 55

Not on the map. Why?

Abstract (arXiv)

A commonly observed problem with the state-of-the art abstractive summarization models is that the generated summaries can be factually inconsistent with the input documents. The fact that automatic summarization may produce plausible-sounding yet inaccurate summaries is a major concern that limits its wide application. In this paper we present an approach to address factual consistency in summarization. We first propose an efficient automatic evaluation metric to measure factual consistency; next, we propose a novel learning algorithm that maximizes the proposed metric during model training. Through extensive experiments, we confirm that our method is effective in improving factual consistency and even overall quality of the summaries, as judged by both automatic metrics and human evaluation.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 10 May 2021.

Builds on

23 of 32 references.

References · 23

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  4. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  7. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  8. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  9. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  10. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  11. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 396
  12. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 378
11 more
  1. 2016NewsQA: A Machine Comprehension Datasetcited by 331
  2. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  3. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 293
  4. 2019Neural Text Summarization: A Critical Evaluationcited by 189
  5. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  6. 2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
  7. 2019Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Trainingcited by 110
  8. 2021Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selectioncited by 76
  9. 2020End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systemscited by 65
  10. 2019A Good Sample is Hard to Find: Noise Injection Sampling and Self-Training for Neural Language Generation Modelscited by 17
  11. 2020Enhancing Factual Consistency of Abstractive Summarizationcited by 10

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 22; 2023: 18; 2024: 7; 2025: 4.

55 papers.

Sort:

By area:

  1. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  2. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 327
  3. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  4. 2021Measuring Attribution in Natural Language Generation Modelscited by 256
  5. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  6. 2022Pretrained Language Models for Text Generation: A Surveycited by 122
  7. 2022Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectorscited by 104
  8. 2022Evaluating Human-Language Model Interactioncited by 92
  9. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  10. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  11. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 72
  12. 2022Generating Literal and Implied Subquestions to Fact-check Complex Claimscited by 66
38 more
  1. 2023FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledgecited by 55
  2. 2022RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Groundingcited by 47
  3. 2022mFACE: Multilingual Summarization with Factual Consistency Evaluationcited by 42
  4. 2022FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representationscited by 41
  5. 2022Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metricscited by 38
  6. 2023MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarizationcited by 30
  7. 2021Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generationcited by 29
  8. 2022Learning to Revise References for Faithful Summarizationcited by 28
  9. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  10. 2023Zero-shot Faithful Factual Error Correctioncited by 26
  11. 2022How Far are We from Robust Long Abstractive Summarization?cited by 25
  12. 2022Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processingcited by 23
  13. 2023Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignmentcited by 19
  14. 2021Explainable Fact-checking through Question Answeringcited by 16
  15. 2023LongDocFACTScore: Evaluating the Factuality of Long Document Abstractive Summarisationcited by 16
  16. 2024Localizing Factual Inconsistencies in Attributable Text Generationcited by 14
  17. 2023Reliability Check: An Analysis of GPT-3's Response to Sensitive Topics and Prompt Wordingcited by 13
  18. 2022Contrastive Error Attribution for Finetuned Language Modelscited by 12
  19. 2023Do Multi-Document Summarization Models Synthesize?cited by 11
  20. 2023SWING: Balancing Coverage and Faithfulness for Dialogue Summarizationcited by 10
  21. 2023An Investigation of Evaluation Metrics for Automated Medical Note Generationcited by 10
  22. 2025Do LLMs Understand Your Translations? Evaluating Paragraph-level MT with Question Answeringcited by 10
  23. 2023Generating EDU Extracts for Plan-Guided Summary Re-Rankingcited by 9
  24. 2022Counterfactual Data Augmentation improves Factuality of Abstractive Summarizationcited by 8
  25. 2023What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarizationcited by 8
  26. 2022Probing Factually Grounded Content Transfer with Factual Ablationcited by 7
  27. 2023Active Learning for Abstractive Text Summarizationcited by 7
  28. 2023Improving Factuality of Abstractive Summarization without Sacrificing Summary Qualitycited by 7
  29. 2024TruthEval: A Dataset to Evaluate LLM Truthfulness and Reliabilitycited by 7
  30. 2022FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustnesscited by 4
  31. 2023P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Modelscited by 4
  32. 2022Consecutive Question Generation via Dynamic Multitask Learningcited by 2
  33. 2023A Dialogue System for Assessing Activities of Daily Living: Improving Consistency with Grounded Knowledgecited by 2
  34. 2024Source Identification in Abstractive Summarizationcited by 2
  35. 2025Fact-Preserved Personalized News Headline Generationcited by 2
  36. 2025Reranking-based Generation for Unbiased Perspective Summarizationcited by 2
  37. 2022Leveraging Key Information Modeling to Improve Less-Data Constrained News Headline Generation via Duality Fine-Tuningcited by 1
  38. 2023On the State of German (Abstractive) Text Summarizationcited by 1

50 most cited of 55 All 55 (JSON)