mlsanityData to 7 Oct 2026

XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages

arXiv 2106.13822 ↗First posted 25 Jun 2021cs.CLcited by 346

Not on the map. Why?

Abstract (arXiv)

Contemporary works on abstractive text summarization have focused primarily on high-resource languages like English, mostly due to the limited availability of datasets for low/mid-resource ones. In this work, we present XL-Sum, a comprehensive and diverse dataset comprising 1 million professionally annotated article-summary pairs from BBC, extracted using a set of carefully designed heuristics. The dataset covers 44 languages ranging from low to high-resource, for many of which no public dataset is currently available. XL-Sum is highly abstractive, concise, and of high quality, as indicated by human and intrinsic evaluation. We fine-tune mT5, a state-of-the-art pretrained multilingual model, with XL-Sum and experiment on multilingual and low-resource summarization tasks. XL-Sum induces competitive results compared to the ones obtained using similar monolingual datasets: we show higher than 11 ROUGE-2 scores on 10 languages we benchmark on, with some of them exceeding 15, as obtained by multilingual training. Additionally, training on low-resource languages individually also provides competitive performance. To the best of our knowledge, XL-Sum is the largest abstractive summarization dataset in terms of the number of samples collected from a single source and the number of languages covered. We are releasing our dataset and models to encourage future research on multilingual abstractive summarization. The resources can be found at urlhttps://github.com/csebuetnlp/xl-sum.

Key facts. Cited by 346 papers in the map (data through 7 Oct 2026). Builds on 31 of its 51 references that are in the map. First posted on arXiv 25 Jun 2021.

Builds on

31 of 51 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  8. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  9. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  10. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  11. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  12. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
19 more
  1. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  2. 2015Teaching Machines to Read and Comprehendcited by 1,039
  3. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  4. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  5. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  6. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  7. 2019Text Summarization with Pretrained Encoderscited by 553
  8. 2017Convolutional Sequence to Sequence Learningcited by 550
  9. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
  10. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  11. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  12. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  13. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  14. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 221
  15. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  16. 2020MLSUM: The Multilingual Summarization Corpuscited by 121
  17. 2019NCLS: Neural Cross-Lingual Summarizationcited by 64
  18. 2018A Unified Model for Extractive and Abstractive Summarization using Inconsistency Losscited by 47
  19. 2020fugashi, a Tool for Tokenizing Japanese in Pythoncited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 35; 2023: 65; 2024: 110; 2025: 84; 2026: 47.

346 papers.

Sort:

By area:

  1. 2023PaLM 2 Technical Reportcited by 1,154
  2. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
  3. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  4. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  5. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  6. 2024Many-Shot In-Context Learningcited by 203
  7. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
  8. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  9. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 163
  10. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
  11. 2024How do Large Language Models Handle Multilingualism?cited by 148
  12. 2024Compact Language Models via Pruning and Knowledge Distillationcited by 142
38 more
  1. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  2. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  3. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  4. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 101
  5. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  6. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 97
  7. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 86
  8. 2023TrueTeacher: Learning Factual Consistency Evaluation with Large Language Modelscited by 82
  9. 2024LLM Pruning and Distillation in Practice: The Minitron Approachcited by 82
  10. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  11. 2023Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarizationcited by 69
  12. 2023BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfercited by 68
  13. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 65
  14. 2024Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performancecited by 61
  15. 2023MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Taskscited by 57
  16. 2024IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languagescited by 55
  17. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  18. 2024LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedbackcited by 51
  19. 2023An Open Dataset and Model for Language Identificationcited by 49
  20. 2023Detecting and Mitigating Hallucinations in Multilingual Summarisationcited by 48
  21. 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 48
  22. 2023AfroBench: How Good are Large Language Models on African Languages?cited by 43
  23. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 43
  24. 2022IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languagescited by 42
  25. 2022mFACE: Multilingual Summarization with Factual Consistency Evaluationcited by 42
  26. 2023SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluationcited by 42
  27. 2024EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Samplingcited by 42
  28. 2024RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMscited by 42
  29. 2024Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performancecited by 41
  30. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41
  31. 2022BanglaNLG and BanglaT5: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Banglacited by 40
  32. 2021IndicBART: A Pre-trained Model for Indic Natural Language Generationcited by 39
  33. 2021CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairscited by 39
  34. 2022EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domaincited by 39
  35. 2022A Survey on Cross-Lingual Summarizationcited by 38
  36. 2022NusaCrowd: Open Source Initiative for Indonesian NLP Resourcescited by 38
  37. 2023GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLPcited by 37
  38. 2023GlotLID: Language Identification for Low-Resource Languagescited by 37

50 most cited of 346 All 346 (JSON)