mlsanityData to 7 Oct 2026

CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization

arXiv 2109.09209 ↗First posted 19 Sep 2021cs.CLcited by 120

Not on the map. Why?

Abstract (arXiv)

We study generating abstractive summaries that are faithful and factually consistent with the given articles. A novel contrastive learning formulation is presented, which leverages both reference summaries, as positive training data, and automatically generated erroneous summaries, as negative training data, to train summarization systems that are better at distinguishing between them. We further design four types of strategies for creating negative samples, to resemble errors made commonly by two state-of-the-art models, BART and PEGASUS, found in our new human annotations of summary errors. Experiments on XSum and CNN/Daily Mail show that our contrastive learning framework is robust across datasets and models. It consistently produces more factual summaries than strong comparisons with post error correction, entailment-based reranking, and unlikelihood training, according to QA-based factuality evaluation. Human judges echo the observation and find that our model summaries correct more errors.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). Builds on 45 of its 58 references that are in the map. First posted on arXiv 19 Sep 2021.

Builds on

45 of 58 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  9. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  10. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  11. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
33 more
  1. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  2. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  3. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  4. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  5. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  6. 2019Text Summarization with Pretrained Encoderscited by 553
  7. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  8. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  9. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  10. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  11. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 285
  12. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  13. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  14. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 241
  15. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  16. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  17. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  18. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 137
  19. 2021Annotating and Modeling Fine-grained Factuality in Summarizationcited by 131
  20. 2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 131
  21. 2020Evaluating Factuality in Generation with Dependency-level Entailmentcited by 122
  22. 2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
  23. 2019Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Trainingcited by 110
  24. 2020Factual Error Correction for Abstractive Summarization Modelscited by 108
  25. 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96
  26. 2019Answers Unite! Unsupervised Metrics for Reinforced Summarization Modelscited by 92
  27. 2021Supporting Clustering with Contrastive Learningcited by 90
  28. 2020GO FIGURE: A Meta Evaluation of Factuality in Summarizationcited by 75
  29. 2021Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selectioncited by 75
  30. 2020Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Rewardcited by 66
  31. 2020Multi-Fact Correction in Abstractive Text Summarizationcited by 66
  32. 2021Focus Attention: Promoting Faithfulness and Diversity in Summarizationcited by 30
  33. 2019Negative Training for Neural Dialogue Response Generationcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 41; 2023: 45; 2024: 16; 2025: 12; 2026: 5.

120 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  3. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  4. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  5. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  6. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 141
  7. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 108
  8. 2022Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectorscited by 103
  9. 2022Evaluating Human-Language Model Interactioncited by 90
  10. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  11. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 71
  12. 2024TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarizationcited by 66
38 more
  1. 2022Cross-modal Contrastive Learning for Speech Translationcited by 61
  2. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  3. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  4. 2022FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarizationcited by 49
  5. 2024Evaluating LLMs at Detecting Errors in LLM Responsescited by 49
  6. 2023Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articlescited by 45
  7. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  8. 2022mFACE: Multilingual Summarization with Factual Consistency Evaluationcited by 42
  9. 2022Contrastive Learning Reduces Hallucination in Conversationscited by 39
  10. 2022Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infillingcited by 38
  11. 2022On Improving Summarization Factual Consistency from Natural Language Feedbackcited by 38
  12. 2022Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarizationcited by 32
  13. 2023Faithfulness-Aware Decoding Strategies for Abstractive Summarizationcited by 31
  14. 2023Teaching Language Models to Hallucinate Less with Synthetic Taskscited by 31
  15. 2022Learning to Revise References for Faithful Summarizationcited by 28
  16. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  17. 2022Towards Improving Faithfulness in Abstractive Summarizationcited by 27
  18. 2023Zero-shot Faithful Factual Error Correctioncited by 26
  19. 2023On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Studycited by 25
  20. 2022Graph Enhanced Contrastive Learning for Radiology Findings Summarizationcited by 24
  21. 2024Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Taskscited by 23
  22. 2022HIBRIDS: Attention with Hierarchical Biases for Structure-aware Long Document Summarizationcited by 22
  23. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
  24. 2022CREATER: CTR-driven Advertising Text Generation with Controlled Pre-Training and Contrastive Fine-Tuningcited by 16
  25. 2022Shortcomings of Question Answering Based Factuality Frameworks for Error Localizationcited by 15
  26. 2023A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarizationcited by 14
  27. 2022Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictionscited by 13
  28. 2024Localizing Factual Inconsistencies in Attributable Text Generationcited by 13
  29. 2025HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detectioncited by 13
  30. 2022Multimedia Generative Script Learning for Task Planningcited by 12
  31. 2022Analyzing and Evaluating Faithfulness in Dialogue Summarizationcited by 12
  32. 2021CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarizationcited by 11
  33. 2023SWING: Balancing Coverage and Faithfulness for Dialogue Summarizationcited by 10
  34. 2023Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarizationcited by 10
  35. 2023Large-Scale and Multi-Perspective Opinion Summarization with Diverse Review Subsetscited by 10
  36. 2023AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generationcited by 10
  37. 2024Evaluating the Factuality of Zero-shot Summarizers Across Varied Domainscited by 10
  38. 2023Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarizationcited by 9

50 most cited of 120 All 120 (JSON)