# Models and Datasets for Cross-Lingual Summarisation

arXiv: https://arxiv.org/abs/2202.09583 · HTML: https://mlsanity.com/paper/2202.09583/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 19 Feb 2022.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [Multilingual Translation with Extensible Multilingual Pretraining and Finetuning](https://mlsanity.com/paper/2008.00401/) (2020)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization](https://mlsanity.com/paper/2010.03093/) (2020)
- [A Survey Of Cross-lingual Word Embedding Models](https://mlsanity.com/paper/1706.04902/) (2017)
- [Hierarchical Transformers for Multi-Document Summarization](https://mlsanity.com/paper/1905.13164/) (2019)
- [MLSUM: The Multilingual Summarization Corpus](https://mlsanity.com/paper/2004.14900/) (2020)
- [Finding Universal Grammatical Relations in Multilingual BERT](https://mlsanity.com/paper/2005.04511/) (2020)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://mlsanity.com/paper/1809.08370/) (2018)
- [NCLS: Neural Cross-Lingual Summarization](https://mlsanity.com/paper/1909.00156/) (2019)
- [Cross-Lingual Natural Language Generation via Pre-Training](https://mlsanity.com/paper/1909.10481/) (2019)
- [WikiAsp: A Dataset for Multi-domain Aspect-based Summarization](https://mlsanity.com/paper/2011.07832/) (2020)
- [Generating Summaries with Topic Templates and Structured Convolutional Decoders](https://mlsanity.com/paper/1906.04687/) (2019)
- [Global Voices: Crossing Borders in Automatic News Summarization](https://mlsanity.com/paper/1910.00421/) (2019)

## Built on by (most cited first, 30 of 36)

- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.16039/) (2023)
- [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://mlsanity.com/paper/2402.06619/) (2024)
- [Bias in Large Language Models: Origin, Evaluation, and Mitigation](https://mlsanity.com/paper/2411.10915/) (2024)
- [EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domain](https://mlsanity.com/paper/2210.13448/) (2022)
- [A Survey on Cross-Lingual Summarization](https://mlsanity.com/paper/2203.12515/) (2022)
- [GEMv2: Multilingual NLG Benchmarking in a Single Line of Code](https://mlsanity.com/paper/2206.11249/) (2022)
- [A Variational Hierarchical Model for Neural Cross-Lingual Summarization](https://mlsanity.com/paper/2203.03820/) (2022)
- [Zero-Shot Cross-Lingual Summarization via Large Language Models](https://mlsanity.com/paper/2302.14229/) (2023)
- [DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine](https://mlsanity.com/paper/2203.10232/) (2022)
- [UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization](https://mlsanity.com/paper/2211.09783/) (2022)
- [Towards Unifying Multi-Lingual and Cross-Lingual Summarization](https://mlsanity.com/paper/2305.09220/) (2023)
- [Exploring Methods for Cross-lingual Text Style Transfer: The Case of Text Detoxification](https://mlsanity.com/paper/2311.13937/) (2023)
- [Understanding Translationese in Cross-Lingual Summarization](https://mlsanity.com/paper/2212.07220/) (2022)
- [Low-Rank Adaptation for Multilingual Summarization: An Empirical Study](https://mlsanity.com/paper/2311.08572/) (2023)
- [InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT](https://mlsanity.com/paper/2305.13083/) (2023)
- [Retrieval-Augmented Machine Translation with Unstructured Knowledge](https://mlsanity.com/paper/2412.04342/) (2024)
- [The Cross-lingual Conversation Summarization Challenge](https://mlsanity.com/paper/2205.00379/) (2022)
- [CroCoSum: A Benchmark Dataset for Cross-Lingual Code-Switched Summarization](https://mlsanity.com/paper/2303.04092/) (2023)
- [μPLAN: Summarizing using a Content Plan as Cross-Lingual Bridge](https://mlsanity.com/paper/2305.14205/) (2023)
- [Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation](https://mlsanity.com/paper/2307.04018/) (2023)
- [WikiDes: A Wikipedia-Based Dataset for Generating Short Descriptions from Paragraphs](https://mlsanity.com/paper/2209.13101/) (2022)
- [Long-Document Cross-Lingual Summarization](https://mlsanity.com/paper/2212.00586/) (2022)
- [WETBench: A Benchmark for Detecting Task-Specific Machine-Generated Text on Wikipedia](https://mlsanity.com/paper/2507.03373/) (2025)
- [Examining Large Pre-Trained Language Models for Machine Translation: What You Don't Know About It](https://mlsanity.com/paper/2209.07417/) (2022)
- [UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Prompt](https://mlsanity.com/paper/2309.11065/) (2023)
- [The State and Fate of Summarization Datasets: A Survey](https://mlsanity.com/paper/2411.04585/) (2024)
- [Multi-Target Cross-Lingual Summarization: a novel task and a language-neutral approach](https://mlsanity.com/paper/2410.00502/) (2024)
- [TSM-Bench: Detecting LLM-Generated Text in Real-World Wikipedia Editing Practices](https://mlsanity.com/paper/2605.31113/) (2026)
- [On the State of German (Abstractive) Text Summarization](https://mlsanity.com/paper/2301.07095/) (2023)

## Abstract (arXiv)

We present a cross-lingual summarisation corpus with long documents in a source language associated with multi-sentence summaries in a target language. The corpus covers twelve language pairs and directions for four European languages, namely Czech, English, French and German, and the methodology for its creation can be applied to several other languages. We derive cross-lingual document-summary instances from Wikipedia by combining lead paragraphs and articles' bodies from language aligned Wikipedia titles. We analyse the proposed cross-lingual summarisation task with automatic metrics and validate it with a human study. To illustrate the utility of our dataset we report experiments with multi-lingual pre-trained models in supervised, zero- and few-shot, and out-of-domain scenarios.