# Camoscio: an Italian Instruction-tuned LLaMA

arXiv: https://arxiv.org/abs/2307.16456 · HTML: https://mlsanity.com/paper/2307.16456/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 95 references that are in the map.
- First posted on arXiv 31 Jul 2023.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Accelerating Transformer Inference for Translation via Parallel Decoding](https://mlsanity.com/paper/2305.10427/) (2023)
- [XFORMAL: A Benchmark for Multilingual Formality Style Transfer](https://mlsanity.com/paper/2104.04108/) (2021)
- [Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering](https://mlsanity.com/paper/2010.12643/) (2020)
- [IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://mlsanity.com/paper/2203.03759/) (2022)
- [GePpeTto Carves Italian into a Language Model](https://mlsanity.com/paper/2004.14253/) (2020)
- [Fauno: The Italian Large Language Model that will leave you senza parole!](https://mlsanity.com/paper/2306.14457/) (2023)

## Built on by (most cited first, 13 of 13)

- [Empowering Cross-lingual Abilities of Instruction-tuned Large Language Models by Translation-following demonstrations](https://mlsanity.com/paper/2308.14186/) (2023)
- [Query Performance Prediction using Relevance Judgments Generated by Large Language Models](https://mlsanity.com/paper/2404.01012/) (2024)
- [A Survey of Large Language Models for European Languages](https://mlsanity.com/paper/2408.15040/) (2024)
- [Disce aut Deficere: Evaluating LLMs Proficiency on the INVALSI Italian Benchmark](https://mlsanity.com/paper/2406.17535/) (2024)
- [The Invalsi Benchmarks: measuring Linguistic and Mathematical understanding of Large Language Models in Italian](https://mlsanity.com/paper/2403.18697/) (2024)
- [Leveraging Large Language Models to Geolocate Linguistic Variations in Social Media Posts](https://mlsanity.com/paper/2407.16047/) (2024)
- [A gentle push funziona benissimo: making instructed models in Italian via contrastive activation steering](https://mlsanity.com/paper/2411.18247/) (2024)
- [CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning](https://mlsanity.com/paper/2506.00875/) (2025)
- [Experimenting AI Technologies for Disinformation Combat: the IDMO Project](https://mlsanity.com/paper/2310.11097/) (2023)
- [Unipa-GPT: Large Language Models for university-oriented QA in Italian](https://mlsanity.com/paper/2407.14246/) (2024)
- [SLIMER-IT: Zero-Shot NER on Italian Language](https://mlsanity.com/paper/2409.15933/) (2024)
- [Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages](https://mlsanity.com/paper/2512.08777/) (2025)
- [Language, Language Models, and What We're Talking About](https://mlsanity.com/paper/2609.03577/) (2026)

## Abstract (arXiv)

In recent years Large Language Models (LLMs) have increased the state of the art on several natural language processing tasks. However, their accessibility is often limited to paid API services, posing challenges for researchers in conducting extensive investigations. On the other hand, while some open-source models have been proposed by the community, they are typically English-centric or multilingual without a specific adaptation for the Italian language. In an effort to democratize the available and open resources for the Italian language, in this paper we introduce Camoscio: a language model specifically tuned to follow users' prompts in Italian. Specifically, we finetuned the smallest variant of LLaMA (7b) with LoRA on a corpus of instruction prompts translated to Italian via ChatGPT. Results indicate that the model's zero-shot performance on various downstream tasks in Italian competes favorably with existing models specifically finetuned for those tasks. All the artifacts (code, dataset, model) are released to the community at the following url: https://github.com/teelinsan/camoscio