mlsanityData to 7 Oct 2026

Advancing Generative AI for Portuguese with Open Decoder Gerv'asio PT*

arXiv 2402.18766 ↗First posted 29 Feb 2024cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

To advance the neural decoding of Portuguese, in this paper we present a fully open Transformer-based, instruction-tuned decoder model that sets a new state of the art in this respect. To develop this decoder, which we named Gerv'asio PT*, a strong LLaMA~2 7B model was used as a starting point, and its further improvement through additional training was done over language resources that include new instruction data sets of Portuguese prepared for this purpose, which are also contributed in this paper. All versions of Gerv'asio are open source and distributed for free under an open license, including for either research or commercial usage, and can be run on consumer-grade hardware, thus seeking to contribute to the advancement of research and innovation in language technology for Portuguese.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

27 of 42 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  10. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
15 more
  1. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  5. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  6. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  7. 2019How multilingual is Multilingual BERT?cited by 642
  8. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  9. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  10. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  11. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  12. 2023Evaluating GPT-3.5 and GPT-4 Models on Brazilian University Admission Examscited by 25
  13. 2023Cabrita: closing the gap for foreign languagescited by 25
  14. 2023Advancing Neural Encoding of Portuguese with Transformer Albertina PT-*cited by 24
  15. 2023BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXamscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 4; 2026: 8.

19 papers.

Sort:

By area:

  1. 2024Tucano: Advancing Neural Text Generation for Portuguesecited by 12
  2. 2024ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Languagecited by 8
  3. 2025Building High-Quality Datasets for Portuguese LLMs: From Common Crawl Snapshots to Industrial-Grade Corporacited by 7
  4. 2024Generative Model for Less-Resourced Language with 1 billion parameterscited by 5
  5. 2024Open Sentence Embeddings for Portuguese with the Serafim PT* encoders familycited by 4
  6. 2025Curi'o-Edu 7B: Examining Data Selection Impacts in LLM Continued Pretrainingcited by 4
  7. 2024Open Generative Large Language Models for Galiciancited by 3
  8. 2026ALBA: A European Portuguese Benchmark for Evaluating Language and Linguistic Dimensions in Generative LLMscited by 3
  9. 2025LegalBench.PT: A Benchmark for Portuguese Lawcited by 2
  10. 2024A Legal Framework for Natural Language Processing Model Training in Portugalcited by 1
  11. 2024Meta-prompting Optimized Retrieval-augmented Generationcited by 1
  12. 2026Building a Strong Instruction Language Model for a Less-Resourced Languagecited by 1
7 more
  1. 2026Sovereign AI-based Public Services are Viable and Affordablecited by 1
  2. 2026CLARIN-PT-LDB: An Open LLM Leaderboard for Portuguese to assess Language, Culture and Civilitycited by 1
  3. 2026P3B3: A Multi-Turn Conversational Benchmark for Measuring European and Brazilian Portuguese Variety Bias in LLMscited by 1
  4. 2025Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Studycited by 0
  5. 2026AMALIA Technical Report: A Fully Open Source Large Language Model for European Portuguesecited by 0
  6. 2026Progressing beyond Art Masterpieces or Touristic Clich'es: how to assess your LLMs for cultural alignment?cited by 0
  7. 2026Language Models for Portuguese: A Systematic Mapping Studycited by 0