mlsanityData to 7 Oct 2026

mFACE: Multilingual Summarization with Factual Consistency Evaluation

arXiv 2212.10622 ↗First posted 20 Dec 2022cs.CLcited by 42

Not on the map. Why?

Abstract (arXiv)

Abstractive summarization has enjoyed renewed interest in recent years, thanks to pre-trained language models and the availability of large-scale datasets. Despite promising results, current models still suffer from generating factually inconsistent summaries, reducing their utility for real-world application. Several recent efforts attempt to address this by devising models that automatically detect factual inconsistencies in machine generated summaries. However, they focus exclusively on English, a language with abundant resources. In this work, we leverage factual consistency evaluation models to improve multilingual summarization. We explore two intuitive approaches to mitigate hallucinations based on the signal provided by a multilingual NLI model, namely data filtering and controlled generation. Experimental results in the 45 languages from the XLSum dataset show gains over strong baselines in both automatic and human evaluation.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 50 of its 69 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

50 of 69 references.

References · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  7. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  8. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  9. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  10. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  11. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
38 more
  1. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  2. 2019Text Summarization with Pretrained Encoderscited by 553
  3. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  4. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  5. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  6. 2020FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarizationcited by 292
  7. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  8. 2019Massively Multilingual Neural Machine Translationcited by 246
  9. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 241
  10. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  11. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
  12. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  13. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
  14. 2017Faithful to the Original: Fact Aware Neural Abstractive Summarizationcited by 141
  15. 2021Annotating and Modeling Fine-grained Factuality in Summarizationcited by 131
  16. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 131
  17. 2018Ranking Sentences for Extractive Summarization with Reinforcement Learningcited by 128
  18. 2020Evaluating Factuality in Generation with Dependency-level Entailmentcited by 122
  19. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  20. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  21. 2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
  22. 2022Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectorscited by 103
  23. 2019Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reportscited by 96
  24. 2020Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summarycited by 87
  25. 2020Improved Natural Language Generation via Loss Truncationcited by 75
  26. 2020GO FIGURE: A Meta Evaluation of Factuality in Summarizationcited by 75
  27. 2021Planning with Learned Entity Prompts for Abstractive Summarizationcited by 75
  28. 2021Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Featurescited by 74
  29. 2018Multi-Reward Reinforced Summarization with Saliency and Entailmentcited by 73
  30. 2019Question Answering as an Automatic Evaluation Metric for News Article Summarizationcited by 63
  31. 2021Improving Factual Consistency of Abstractive Summarization via Question Answeringcited by 55
  32. 2022FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarizationcited by 49
  33. 2022Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusterscited by 44
  34. 2021Focus Attention: Promoting Faithfulness and Diversity in Summarizationcited by 30
  35. 2022A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generationcited by 28
  36. 2018Structure-Infused Copy Mechanisms for Abstractive Summarizationcited by 17
  37. 2019Guiding Extractive Summarization with Question-Answering Rewardscited by 13
  38. 2020Multi-Style Transfer with Discriminative Feedback on Disjoint Corpuscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 19; 2025: 6; 2026: 2.

42 papers.

Sort:

By area:

  1. 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 395
  2. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  3. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  4. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 105
  5. 2023TrueTeacher: Learning Factual Consistency Evaluation with Large Language Modelscited by 82
  6. 2023FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledgecited by 55
  7. 2023From Sparse to Dense: GPT-4 Summarization with Chain of Density Promptingcited by 52
  8. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  9. 2023Detecting and Mitigating Hallucinations in Multilingual Summarisationcited by 48
  10. 2023SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluationcited by 42
  11. 2024EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Samplingcited by 42
  12. 2024K-QA: A Real-World Medical Q&A Benchmarkcited by 30
30 more
  1. 2026Hidden in Memory: Sleeper Memory Poisoning in LLM Agentscited by 22
  2. 2023Evaluating and Modeling Attribution for Cross-Lingual Question Answeringcited by 21
  3. 2024TrustScore: Reference-Free Evaluation of LLM Response Trustworthinesscited by 19
  4. 2024Comparing Hallucination Detection Metrics for Multilingual Generationcited by 18
  5. 2024Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScorecited by 18
  6. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  7. 2023Low-Rank Adaptation for Multilingual Summarization: An Empirical Studycited by 13
  8. 2024Learning to Generate Answers with Citations via Factual Consistency Modelscited by 13
  9. 2025Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languagescited by 10
  10. 2023KL-Divergence Guided Temperature Samplingcited by 9
  11. 2024XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tagscited by 9
  12. 2024SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Maskscited by 9
  13. 2023μPLAN: Summarizing using a Content Plan as Cross-Lingual Bridgecited by 7
  14. 2024Hierarchical Indexing for Retrieval-Augmented Opinion Summarizationcited by 7
  15. 2025How Much Do LLMs Hallucinate across Languages? On Realistic Multilingual Estimation of LLM Hallucinationcited by 5
  16. 2023Calibrating Likelihoods towards Consistency in Summarization Modelscited by 4
  17. 2023P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Modelscited by 4
  18. 2024Selected Languages are All You Need for Cross-lingual Truthfulness Transfercited by 4
  19. 2024EvoPat: A Multi-LLM-based Patents Summarization and Analysis Agentcited by 4
  20. 2026Cross-Lingual LLM-Judge Transfer via Evaluation Decompositioncited by 3
  21. 2024Do Zombies Understand? A Choose-Your-Own-Adventure Exploration of Machine Cognitioncited by 2
  22. 2024German also Hallucinates! Inconsistency Detection in News Summaries with the Absinth Datasetcited by 2
  23. 2024A Critical Look at Meta-evaluating Summarisation Evaluation Metricscited by 2
  24. 2025Evaluating the Evaluator: Summarization Metrics and LLM-Judges beyond Englishcited by 2
  25. 2025A Generative-AI-Driven Claim Retrieval System Capable of Detecting and Retrieving Claims from Social Media Platforms in Multiple Languagescited by 1
  26. 2025Multilingual Self-Taught Faithfulness Evaluatorscited by 1
  27. 2024Improving Factual Error Correction for Abstractive Summarization via Data Distillation and Conditional-generation Clozecited by 0
  28. 2024Multilingual Fine-Grained News Headline Hallucination Detectioncited by 0
  29. 2024Leveraging Entailment Judgements in Cross-Lingual Summarisationcited by 0
  30. 2025An Empirical Study of Many-to-Many Summarization with Large Language Modelscited by 0