# mFACE: Multilingual Summarization with Factual Consistency Evaluation

arXiv: https://arxiv.org/abs/2212.10622 · HTML: https://mlsanity.com/paper/2212.10622/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 69 references that are in the map.
- First posted on arXiv 20 Dec 2022.

## Builds on (most cited first, 30 of 50)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization](https://mlsanity.com/paper/2005.03754/) (2020)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1903.00089/) (2019)
- [QuestEval: Summarization Asks for Fact-based Evaluation](https://mlsanity.com/paper/2103.12693/) (2021)
- [The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics](https://mlsanity.com/paper/2102.01672/) (2021)
- [QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization](https://mlsanity.com/paper/2112.08542/) (2021)
- [Scaling Up Models and Data with textttt5x and textttseqio](https://mlsanity.com/paper/2203.17189/) (2022)
- [Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text](https://mlsanity.com/paper/2202.06935/) (2022)
- [Faithful to the Original: Fact Aware Neural Abstractive Summarization](https://mlsanity.com/paper/1711.04434/) (2017)
- [Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering](https://mlsanity.com/paper/2104.08202/) (2021)
- [Annotating and Modeling Fine-grained Factuality in Summarization](https://mlsanity.com/paper/2104.04302/) (2021)
- [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://mlsanity.com/paper/1802.08636/) (2018)
- [Evaluating Factuality in Generation with Dependency-level Entailment](https://mlsanity.com/paper/2010.05478/) (2020)

## Built on by (most cited first, 30 of 42)

- [SLiC-HF: Sequence Likelihood Calibration with Human Feedback](https://mlsanity.com/paper/2305.10425/) (2023)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [What You See is What You Read? Improving Text-Image Alignment Evaluation](https://mlsanity.com/paper/2305.10400/) (2023)
- [TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models](https://mlsanity.com/paper/2305.11171/) (2023)
- [FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge](https://mlsanity.com/paper/2305.08281/) (2023)
- [From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting](https://mlsanity.com/paper/2309.04269/) (2023)
- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [Detecting and Mitigating Hallucinations in Multilingual Summarisation](https://mlsanity.com/paper/2305.13632/) (2023)
- [EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling](https://mlsanity.com/paper/2403.14541/) (2024)
- [SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation](https://mlsanity.com/paper/2305.13194/) (2023)
- [K-QA: A Real-World Medical Q&A Benchmark](https://mlsanity.com/paper/2401.14493/) (2024)
- [Hidden in Memory: Sleeper Memory Poisoning in LLM Agents](https://mlsanity.com/paper/2605.15338/) (2026)
- [Evaluating and Modeling Attribution for Cross-Lingual Question Answering](https://mlsanity.com/paper/2305.14332/) (2023)
- [TrustScore: Reference-Free Evaluation of LLM Response Trustworthiness](https://mlsanity.com/paper/2402.12545/) (2024)
- [Comparing Hallucination Detection Metrics for Multilingual Generation](https://mlsanity.com/paper/2402.10496/) (2024)
- [Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScore](https://mlsanity.com/paper/2402.18045/) (2024)
- [Teaching LLMs to Abstain across Languages via Multilingual Feedback](https://mlsanity.com/paper/2406.15948/) (2024)
- [Low-Rank Adaptation for Multilingual Summarization: An Empirical Study](https://mlsanity.com/paper/2311.08572/) (2023)
- [Learning to Generate Answers with Citations via Factual Consistency Models](https://mlsanity.com/paper/2406.13124/) (2024)
- [Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages](https://mlsanity.com/paper/2506.00549/) (2025)
- [KL-Divergence Guided Temperature Sampling](https://mlsanity.com/paper/2306.01286/) (2023)
- [XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags](https://mlsanity.com/paper/2406.03776/) (2024)
- [SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks](https://mlsanity.com/paper/2410.05102/) (2024)
- [μPLAN: Summarizing using a Content Plan as Cross-Lingual Bridge](https://mlsanity.com/paper/2305.14205/) (2023)
- [Hierarchical Indexing for Retrieval-Augmented Opinion Summarization](https://mlsanity.com/paper/2403.00435/) (2024)
- [How Much Do LLMs Hallucinate across Languages? On Realistic Multilingual Estimation of LLM Hallucination](https://mlsanity.com/paper/2502.12769/) (2025)
- [Calibrating Likelihoods towards Consistency in Summarization Models](https://mlsanity.com/paper/2310.08764/) (2023)
- [P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models](https://mlsanity.com/paper/2311.09741/) (2023)
- [Selected Languages are All You Need for Cross-lingual Truthfulness Transfer](https://mlsanity.com/paper/2406.14434/) (2024)

## Abstract (arXiv)

Abstractive summarization has enjoyed renewed interest in recent years, thanks to pre-trained language models and the availability of large-scale datasets. Despite promising results, current models still suffer from generating factually inconsistent summaries, reducing their utility for real-world application. Several recent efforts attempt to address this by devising models that automatically detect factual inconsistencies in machine generated summaries. However, they focus exclusively on English, a language with abundant resources. In this work, we leverage factual consistency evaluation models to improve multilingual summarization. We explore two intuitive approaches to mitigate hallucinations based on the signal provided by a multilingual NLI model, namely data filtering and controlled generation. Experimental results in the 45 languages from the XLSum dataset show gains over strong baselines in both automatic and human evaluation.