# Towards Unifying Multi-Lingual and Cross-Lingual Summarization

arXiv: https://arxiv.org/abs/2305.09220 · HTML: https://mlsanity.com/paper/2305.09220/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 16 May 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization](https://mlsanity.com/paper/2010.03093/) (2020)
- [MLSUM: The Multilingual Summarization Corpus](https://mlsanity.com/paper/2004.14900/) (2020)
- [The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey](https://mlsanity.com/paper/2104.14839/) (2021)
- [NCLS: Neural Cross-Lingual Summarization](https://mlsanity.com/paper/1909.00156/) (2019)
- [Cross-Lingual Natural Language Generation via Pre-Training](https://mlsanity.com/paper/1909.10481/) (2019)
- [PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization](https://mlsanity.com/paper/2110.08499/) (2021)
- [MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs](https://mlsanity.com/paper/2104.08692/) (2021)
- [A Survey on Cross-Lingual Summarization](https://mlsanity.com/paper/2203.12515/) (2022)
- [Models and Datasets for Cross-Lingual Summarisation](https://mlsanity.com/paper/2202.09583/) (2022)
- [ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization](https://mlsanity.com/paper/2202.05599/) (2022)
- [Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity Alignment](https://mlsanity.com/paper/2304.01563/) (2023)
- [Mixed-Lingual Pre-training for Cross-lingual Summarization](https://mlsanity.com/paper/2010.08892/) (2020)
- [A Variational Hierarchical Model for Neural Cross-Lingual Summarization](https://mlsanity.com/paper/2203.03820/) (2022)
- [Zero-Shot Cross-Lingual Summarization via Large Language Models](https://mlsanity.com/paper/2302.14229/) (2023)
- [Understanding Translationese in Cross-Lingual Summarization](https://mlsanity.com/paper/2212.07220/) (2022)
- [Summary-Oriented Vision Modeling for Multimodal Abstractive Summarization](https://mlsanity.com/paper/2212.07672/) (2022)
- [Analyzing and Evaluating Faithfulness in Dialogue Summarization](https://mlsanity.com/paper/2210.11777/) (2022)
- [The Cross-lingual Conversation Summarization Challenge](https://mlsanity.com/paper/2205.00379/) (2022)
- [Long-Document Cross-Lingual Summarization](https://mlsanity.com/paper/2212.00586/) (2022)

## Built on by (most cited first, 15 of 15)

- [Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages](https://mlsanity.com/paper/2310.14799/) (2023)
- [Cross-Lingual Knowledge Editing in Large Language Models](https://mlsanity.com/paper/2309.08952/) (2023)
- [GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization](https://mlsanity.com/paper/2410.04087/) (2024)
- [Multi-Modal Knowledge Graph Transformer Framework for Multi-Modal Entity Alignment](https://mlsanity.com/paper/2310.06365/) (2023)
- [AspectMMKG: A Multi-modal Knowledge Graph with Aspect-aware Entities](https://mlsanity.com/paper/2308.04992/) (2023)
- [MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models](https://mlsanity.com/paper/2402.13606/) (2024)
- [Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide](https://mlsanity.com/paper/2411.09539/) (2024)
- [Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs](https://mlsanity.com/paper/2502.14645/) (2025)
- [D2TV: Dual Knowledge Distillation and Target-oriented Vision Modeling for Many-to-Many Multimodal Summarization](https://mlsanity.com/paper/2305.12767/) (2023)
- [Multi-Target Cross-Lingual Summarization: a novel task and a language-neutral approach](https://mlsanity.com/paper/2410.00502/) (2024)
- [Snowman: A Million-scale Chinese Commonsense Knowledge Graph Distilled from Foundation Model](https://mlsanity.com/paper/2306.10241/) (2023)
- [SumTra: A Differentiable Pipeline for Few-Shot Cross-Lingual Summarization](https://mlsanity.com/paper/2403.13240/) (2024)
- [Towards Consistency Filtering-Free Unsupervised Learning for Dense Retrieval](https://mlsanity.com/paper/2308.02926/) (2023)
- [MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models](https://mlsanity.com/paper/2410.12478/) (2024)
- [An Empirical Study of Many-to-Many Summarization with Large Language Models](https://mlsanity.com/paper/2505.12983/) (2025)

## Abstract (arXiv)

To adapt text summarization to the multilingual world, previous work proposes multi-lingual summarization (MLS) and cross-lingual summarization (CLS). However, these two tasks have been studied separately due to the different definitions, which limits the compatible and systematic research on both of them. In this paper, we aim to unify MLS and CLS into a more general setting, i.e., many-to-many summarization (M2MS), where a single model could process documents in any language and generate their summaries also in any language. As the first step towards M2MS, we conduct preliminary studies to show that M2MS can better transfer task knowledge across different languages than MLS and CLS. Furthermore, we propose Pisces, a pre-trained M2MS model that learns language modeling, cross-lingual ability and summarization ability via three-stage pre-training. Experimental results indicate that our Pisces significantly outperforms the state-of-the-art baselines, especially in the zero-shot directions, where there is no training data from the source-language documents to the target-language summaries.