mlsanityData to 14 Oct 2026

Introducing Bode: A Fine-Tuned Large Language Model for Portuguese Prompt-Based Task

arXiv 2401.02909 ↗First posted 5 Jan 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are increasingly bringing advances to Natural Language Processing. However, low-resource languages, those lacking extensive prominence in datasets for various NLP tasks, or where existing datasets are not as substantial, such as Portuguese, already obtain several benefits from LLMs, but not to the same extent. LLMs trained on multilingual datasets normally struggle to respond to prompts in Portuguese satisfactorily, presenting, for example, code switching in their responses. This work proposes a fine-tuned LLaMA 2-based model for Portuguese prompts named Bode in two versions: 7B and 13B. We evaluate the performance of this model in classification tasks using the zero-shot approach with in-context learning, and compare it with other LLMs. Our main contribution is to bring an LLM with satisfactory results in the Portuguese language, as well as to provide a model that is free for research or commercial purposes.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 13 of its 18 references that are in the map. First posted on arXiv 5 Jan 2024.

Builds on

13 of 18 references.

References · 13

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2020Longformer: The Long-Document Transformercited by 3,413
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2020GLU Variants Improve Transformercited by 1,804
  8. 2019Root Mean Square Layer Normalizationcited by 1,480
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  11. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  12. 2023Cabrita: closing the gap for foreign languagescited by 25
1 more
  1. 2017Building a Sentiment Corpus of Tweets in Brazilian Portuguesecited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 3; 2026: 1.

12 papers.

  1. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  2. 2024TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguesecited by 13
  3. 2024Tucano: Advancing Neural Text Generation for Portuguesecited by 13
  4. 2024ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Languagecited by 8
  5. 2024LlamaTurk: Adapting Open-Source Generative Large Language Models for Low-Resource Languagecited by 6
  6. 2024A Survey of Large Language Models for European Languagescited by 6
  7. 2024SLIM-RAFT: A Novel Fine-Tuning Approach to Improve Cross-Linguistic Performance for Mercosur Common Nomenclaturecited by 3
  8. 2024Open Llama2 Model for the Lithuanian Languagecited by 2
  9. 2025CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuningcited by 1
  10. 2025Zero-shot Performance of Generative AI in Brazilian Portuguese Medical Examcited by 0
  11. 2025Local LLM Ensembles for Zero-shot Portuguese Named Entity Recognitioncited by 0
  12. 2026Language Models for Portuguese: A Systematic Mapping Studycited by 0