mlsanityData to 7 Oct 2026

PTT5: Pretraining and validating the T5 model on Brazilian Portuguese data

arXiv 2008.09144 ↗First posted 20 Aug 2020cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

In natural language processing (NLP), there is a need for more resources in Portuguese, since much of the data used in the state-of-the-art research is in other languages. In this paper, we pretrain a T5 model on the BrWac corpus, an extensive collection of web pages in Portuguese, and evaluate its performance against other Portuguese pretrained models and multilingual models on three different tasks. We show that our Portuguese pretrained models have significantly better performance over the original T5 models. Moreover, we demonstrate the positive impact of using a Portuguese vocabulary. Our code and models are available at https://github.com/unicamp-dl/PTT5.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 13; 2023: 7; 2024: 6; 2025: 1; 2026: 3.

35 papers.

Sort:

  1. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  2. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  3. 2023Cabrita: closing the gap for foreign languagescited by 25
  4. 2022IT5: Text-to-text Pretraining for Italian Language Understanding and Generationcited by 20
  5. 2023Sabi'a: Portuguese Large Language Modelscited by 19
  6. 2024Gl'orIA -- A Generative and Open Large Language Model for Portuguesecited by 16
  7. 2022Evaluation of Transfer Learning for Polish with a Text-to-Text Modelcited by 13
  8. 2024Tucano: Advancing Neural Text Generation for Portuguesecited by 12
  9. 2022Billions of Parameters Are Worth More Than In-domain Training Data: A case study in the Legal Case Entailment Taskcited by 8
  10. 2024ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Languagecited by 8
  11. 2021A cost-benefit analysis of cross-lingual transfer methodscited by 7
  12. 2024Text Generation Models for Luxembourgish with Limited Data: A Balanced Multilingual Strategycited by 7
23 more
  1. 2021DEEPAG'E: Answering Questions in Portuguese about the Brazilian Environmentcited by 5
  2. 2021PLSUM: Generating PT-BR Wikipedia by Summarizing Multiple Websitescited by 2
  3. 2022Sequence-to-Sequence Models for Extracting Information from Registration and Legal Documentscited by 2
  4. 2022Integrating question answering and text-to-SQL in Portuguesecited by 2
  5. 2022Tweets2Stance: Users stance detection exploiting Zero-Shot Learning Algorithms on Tweetscited by 2
  6. 2022Detecting Languages Unintelligible to Multilingual Models through Local Structure Probescited by 2
  7. 2022Embedding generation for text classification of Brazilian Portuguese user reviews: from bag-of-words to transformerscited by 2
  8. 2023Sequence-to-Sequence Spanish Pre-trained Language Modelscited by 2
  9. 2023DeBERTinha: A Multistep Approach to Adapt DebertaV3 XSmall for Brazilian Portuguese Natural Language Processing Taskcited by 2
  10. 2024Evaluating Named Entity Recognition: A comparative analysis of mono- and multilingual transformer models on a novel Brazilian corporate earnings call transcripts datasetcited by 2
  11. 2022BERT for Sentiment Analysis: Pre-trained and Fine-Tuned Alternativescited by 1
  12. 2022MonoByte: A Pool of Monolingual Byte-level Language Modelscited by 1
  13. 2022Local Structure Matters Most in Most Languagescited by 1
  14. 2023Benchmarks for Pir'a 2.0, a Reading Comprehension Dataset about the Ocean, the Brazilian Coast, and Climate Changecited by 1
  15. 2023Deep Learning Brasil at ABSAPT 2022: Portuguese Transformer Ensemble Approachescited by 1
  16. 2024FairytaleQA Translated: Enabling Educational Question and Answer Generation in Less-Resourced Languagescited by 1
  17. 2022The BLue Amazon Brain (BLAB): A Modular Architecture of Services about the Brazilian Maritime Territorycited by 0
  18. 2022BLAB Reporter: Automated journalism covering the Blue Amazoncited by 0
  19. 2023Abstractive Text Summarization for Resumes With Cutting Edge NLP Transformers and LSTMcited by 0
  20. 2025AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalizationcited by 0
  21. 2026CitiLink-Summ: Summarization of Discussion Subjects in European Portuguese Municipal Meeting Minutescited by 0
  22. 2026Misinformation Span Detection in Videos via Audio Transcriptscited by 0
  23. 2026Language Models for Portuguese: A Systematic Mapping Studycited by 0