mlsanityData to 7 Oct 2026

mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences

arXiv 2305.11129 ↗First posted 18 May 2023cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

We present our work on developing a multilingual, efficient text-to-text transformer that is suitable for handling long inputs. This model, called mLongT5, builds upon the architecture of LongT5, while leveraging the multilingual datasets used for pretraining mT5 and the pretraining tasks of UL2. We evaluate this model on a variety of multilingual summarization and question-answering tasks, and the results show stronger performance for mLongT5 when compared to existing multilingual models such as mBART or M-BERT.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

15 of 18 references.

References · 15

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2020Longformer: The Long-Document Transformercited by 3,362
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  5. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  6. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  8. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  9. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  10. 2022UL2: Unifying Language Learning Paradigmscited by 292
  11. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  12. 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
3 more
  1. 2020WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarizationcited by 154
  2. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  3. 2020MLSUM: The Multilingual Summarization Corpuscited by 121

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 5.

10 papers.

  1. 2024A Survey of Sustainability in Large Language Models: Applications, Economics, and Challengescited by 18
  2. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  3. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 15
  4. 2024An Open Multilingual System for Scoring Readability of Wikipediacited by 7
  5. 2025When LLMs Struggle: Reference-less Translation Evaluation for Low-resource Languagescited by 7
  6. 2024HeSum: a Novel Dataset for Abstractive Text Summarization in Hebrewcited by 6
  7. 2024German also Hallucinates! Inconsistency Detection in News Summaries with the Absinth Datasetcited by 2
  8. 2024Granularity is crucial when applying differential privacy to text: An investigation for neural machine translationcited by 1
  9. 2025AIstorian lets AI be a historian: A KG-powered multi-agent system for accurate biography generationcited by 1
  10. 2025The Role of Orthographic Consistency in Multilingual Embedding Models for Text Classification in Arabic-Script Languagescited by 0