mlsanityData to 7 Oct 2026

Advanced Natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA

arXiv 2405.07101 ↗First posted 11 May 2024cs.CL cs.AIcited by 17

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

In the pursuit of advancing natural language processing for the Italian language, we introduce a state-of-the-art Large Language Model (LLM) based on the novel Meta LLaMA-3 model: LLaMAntino-3-ANITA-8B-Inst-DPO-ITA. We fine-tuned the original 8B parameters instruction tuned model using the Supervised Fine-tuning (SFT) technique on the English and Italian language datasets in order to improve the original performance. Consequently, a Dynamic Preference Optimization (DPO) process has been used to align preferences, avoid dangerous and inappropriate answers, and limit biases and prejudices. Our model leverages the efficiency of QLoRA to fine-tune the model on a smaller portion of the original model weights and then adapt the model specifically for the Italian linguistic structure, achieving significant improvements in both performance and computational efficiency. Concurrently, DPO is employed to refine the model's output, ensuring that generated content aligns with quality answers. The synergy between SFT, QLoRA's parameter efficiency and DPO's user-centric optimization results in a robust LLM that excels in a variety of tasks, including but not limited to text completion, zero-shot classification, and contextual understanding. The model has been extensively evaluated over standard benchmarks for the Italian and English languages, showing outstanding results. The model is freely available over the HuggingFace hub and, examples of use can be found in our GitHub repository. https://huggingface.co/swap-uniba/LLaMAntino-3-ANITA-8B-Inst-DPO-ITA

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 40 of its 78 references that are in the map. First posted on arXiv 11 May 2024.

Builds on

40 of 78 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024ORPO: Monolithic Preference Optimization without Reference Model12 Mar
  2. 2023A General Theoretical Paradigm to Understand Learning from Human Preferences18 Oct
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  4. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  12. 2023Mistral 7Bcited by 3,065
24 more
  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  2. 2023A Survey of Large Language Modelscited by 2,537
  3. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  4. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  5. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  6. 2020Aligning AI With Shared Human Valuescited by 706
  7. 2023A Comprehensive Overview of Large Language Modelscited by 639
  8. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  9. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  10. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  11. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
  12. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  13. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  14. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  15. 2019A Question-Entailment Approach to Question Answeringcited by 133
  16. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  17. 2021XFORMAL: A Benchmark for Multilingual Formality Style Transfercited by 51
  18. 2023Learning from models beyond fine-tuningcited by 47
  19. 2022Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialogcited by 30
  20. 2023LLaMAntino: LLaMA 2 Models for Effective Text Generation in Italian Languagecited by 24
  21. 2023Fauno: The Italian Large Language Model that will leave you senza parole!cited by 13
  22. 2023Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Modelscited by 13
  23. 2017Model Transfer for Tagging Low-resource Languages using a Bilingual Dictionarycited by 10
  24. 2023Cerbero-7B: A Leap Forward in Language-Specific LLMs Through Enhanced Chat Corpus Generation and Evaluationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 5; 2026: 5.

17 papers.

Sort:

By area:

  1. 2025Evalita-LLM: Benchmarking Large Language Models on Italiancited by 6
  2. 2024Disce aut Deficere: Evaluating LLMs Proficiency on the INVALSI Italian Benchmarkcited by 5
  3. 2025Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AIcited by 3
  4. 2025Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settingscited by 3
  5. 2024Leveraging Large Language Models to Geolocate Linguistic Variations in Social Media Postscited by 2
  6. 2025Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptationcited by 2
  7. 2024Automatic identification of diagnosis from hospital discharge letters via weakly supervised Natural Language Processingcited by 1
  8. 2024A gentle push funziona benissimo: making instructed models in Italian via contrastive activation steeringcited by 1
  9. 2026Physical Commonsense Reasoning for Lower-Resourced Languages and Dialects: a Study on Basquecited by 1
  10. 2024Unipa-GPT: Large Language Models for university-oriented QA in Italiancited by 0
  11. 2024SLIMER-IT: Zero-Shot NER on Italian Languagecited by 0
  12. 2024Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustmentscited by 0
5 more
  1. 2025From User Preferences to Optimization Constraints Using Large Language Modelscited by 0
  2. 2026Math Education Digital Shadows for Investigating Learning with GenAI: Mathematics Performance, Anxiety, and Confidence in LLMscited by 0
  3. 2026Natural Language Processing Psychometricscited by 0
  4. 2026Language, Language Models, and What We're Talking Aboutcited by 0
  5. 2026Deconstructing Stereotypes: Scope-Conditioned Generation for Effective Multilingual Counterspeechcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Challenging the Abilities of Large Language Models in Italian: a Community Initiative
  2. 2025CM-Align: Consistency-based Multilingual Alignment for Large Language Models
  3. 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities
  4. 2025Qomhra: A Bilingual Irish and English Large Language Model
  5. 2024PAFT: A Parallel Training Paradigm for Effective LLM Fine-Tuning
  6. 2025InfoPO: On Mutual Information Maximization for Large Language Model Alignment
  7. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers
  8. 2026An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models