# LLaMAntino: LLaMA 2 Models for Effective Text Generation in Italian Language

arXiv: https://arxiv.org/abs/2312.09993 · HTML: https://mlsanity.com/paper/2312.09993/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models](https://mlsanity.com/paper/2102.02503/) (2021)
- [Towards a Cleaner Document-Oriented Multilingual Crawled Corpus](https://mlsanity.com/paper/2201.06642/) (2022)
- [BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting](https://mlsanity.com/paper/2212.09535/) (2022)
- [A Survey on Methods and Theories of Quantized Neural Networks](https://mlsanity.com/paper/1808.04752/) (2018)
- [LLM-FP4: 4-Bit Floating-Point Quantized Transformers](https://mlsanity.com/paper/2310.16836/) (2023)
- [IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://mlsanity.com/paper/2203.03759/) (2022)

## Built on by (most cited first, 25 of 25)

- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [Advanced Natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA](https://mlsanity.com/paper/2405.07101/) (2024)
- [Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment](https://mlsanity.com/paper/2602.16660/) (2026)
- [RoQLlama: A Lightweight Romanian Adapted Language Model](https://mlsanity.com/paper/2410.04269/) (2024)
- [From Bytes to Borsch: Fine-Tuning Gemma and Mistral for the Ukrainian Language Representation](https://mlsanity.com/paper/2404.09138/) (2024)
- [A Survey of Large Language Models for European Languages](https://mlsanity.com/paper/2408.15040/) (2024)
- [Disce aut Deficere: Evaluating LLMs Proficiency on the INVALSI Italian Benchmark](https://mlsanity.com/paper/2406.17535/) (2024)
- [From English-Centric to Effective Bilingual: LLMs with Custom Tokenizers for Underrepresented Languages](https://mlsanity.com/paper/2410.18836/) (2024)
- [The Invalsi Benchmarks: measuring Linguistic and Mathematical understanding of Large Language Models in Italian](https://mlsanity.com/paper/2403.18697/) (2024)
- [Fine-tuning with HED-IT: The impact of human post-editing for dialogical language models](https://mlsanity.com/paper/2406.07288/) (2024)
- [Speech Analysis of Language Varieties in Italy](https://mlsanity.com/paper/2406.15862/) (2024)
- ["Vorbecsti Romanecste?" A Recipe to Train Powerful Romanian LLMs with English Instructions](https://mlsanity.com/paper/2406.18266/) (2024)
- [LLMs for Extremely Low-Resource Finno-Ugric Languages](https://mlsanity.com/paper/2410.18902/) (2024)
- [Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settings](https://mlsanity.com/paper/2505.15353/) (2025)
- [Leveraging Large Language Models to Geolocate Linguistic Variations in Social Media Posts](https://mlsanity.com/paper/2407.16047/) (2024)
- [Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation](https://mlsanity.com/paper/2504.17025/) (2025)
- [Evaluation of Few-Shot Learning for Classification Tasks in the Polish Language](https://mlsanity.com/paper/2404.17832/) (2024)
- [A gentle push funziona benissimo: making instructed models in Italian via contrastive activation steering](https://mlsanity.com/paper/2411.18247/) (2024)
- [Igea: a Decoder-Only Language Model for Biomedical Text Generation in Italian](https://mlsanity.com/paper/2407.06011/) (2024)
- [Unipa-GPT: Large Language Models for university-oriented QA in Italian](https://mlsanity.com/paper/2407.14246/) (2024)
- [From User Preferences to Optimization Constraints Using Large Language Models](https://mlsanity.com/paper/2503.21360/) (2025)
- [epiGPTope: A machine learning-based epitope generator and classifier](https://mlsanity.com/paper/2509.03351/) (2025)
- [Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs](https://mlsanity.com/paper/2509.13480/) (2025)
- [DIETA: A Decoder-only transformer-based model for Italian-English machine TrAnslation](https://mlsanity.com/paper/2601.17823/) (2026)
- [The TEA Nets framework combines AI and cognitive network science to model targets, events and actors in text](https://mlsanity.com/paper/2604.27673/) (2026)

## Abstract (arXiv)

Large Language Models represent state-of-the-art linguistic models designed to equip computers with the ability to comprehend natural language. With its exceptional capacity to capture complex contextual relationships, the LLaMA (Large Language Model Meta AI) family represents a novel advancement in the field of natural language processing by releasing foundational models designed to improve the natural language understanding abilities of the transformer architecture thanks to their large amount of trainable parameters (7, 13, and 70 billion parameters). In many natural language understanding tasks, these models obtain the same performances as private company models such as OpenAI Chat-GPT with the advantage to make publicly available weights and code for research and commercial uses. In this work, we investigate the possibility of Language Adaptation for LLaMA models, explicitly focusing on addressing the challenge of Italian Language coverage. Adopting an open science approach, we explore various tuning approaches to ensure a high-quality text generated in Italian suitable for common tasks in this underrepresented language in the original models' datasets. We aim to release effective text generation models with strong linguistic properties for many tasks that seem challenging using multilingual or general-purpose LLMs. By leveraging an open science philosophy, this study contributes to Language Adaptation strategies for the Italian language by introducing the novel LLaMAntino family of Italian LLMs.