# Small LLMs for Medical NLP: a Systematic Analysis of Few-Shot, Constraint Decoding, Fine-Tuning and Continual Pre-Training in Italian

arXiv: https://arxiv.org/abs/2602.17475 · HTML: https://mlsanity.com/paper/2602.17475/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning](https://mlsanity.com/paper/2305.13971/) (2023)
- [Continual Pre-Training Mitigates Forgetting in Language and Vision](https://mlsanity.com/paper/2205.09357/) (2022)
- [Investigating Continual Pretraining in Large Language Models: Insights and Implications](https://mlsanity.com/paper/2402.17400/) (2024)
- [Enhancing Training Efficiency Using Packing with Flash Attention](https://mlsanity.com/paper/2407.09105/) (2024)
- [Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque](https://mlsanity.com/paper/2506.07597/) (2025)
- [CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures](https://mlsanity.com/paper/2410.05235/) (2024)
- [Converting Annotated Clinical Cases into Structured Case Report Forms](https://mlsanity.com/paper/2506.11666/) (2025)
- [Advancing Italian Biomedical Information Extraction with Transformers-based Models: Methodological Insights and Multicenter Practical Application](https://mlsanity.com/paper/2306.05323/) (2023)
- [Low-resource Information Extraction with the European Clinical Case Corpus](https://mlsanity.com/paper/2503.20568/) (2025)

## Built on by (most cited first, 3 of 3)

- [LLM StructCore: Schema-Guided Reasoning Condensation and Deterministic Compilation](https://mlsanity.com/paper/2604.20560/) (2026)
- [DECSELFMASK: Leveraging Unlabeled Text via Self-Relevance-Guided Masking for Decoder-Only Classification](https://mlsanity.com/paper/2606.09466/) (2026)
- [eCREAM-MedCorpus A Large-Scale Corpus of Clinical Notes for Italian](https://mlsanity.com/paper/2606.12569/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) consistently excel in diverse medical Natural Language Processing (NLP) tasks, yet their substantial computational requirements often limit deployment in real-world healthcare settings. In this work, we investigate whether "small" LLMs (around one billion parameters) can effectively perform medical tasks while maintaining competitive accuracy. We evaluate models from three major families-Llama-3, Gemma-3, and Qwen3-across 20 clinical NLP tasks among Named Entity Recognition, Relation Extraction, Case Report Form Filling, Question Answering, and Argument Mining. We systematically compare a range of adaptation strategies, both at inference time (few-shot prompting, constraint decoding) and at training time (supervised fine-tuning, continual pretraining). Fine-tuning emerges as the most effective approach, while the combination of few-shot prompting and constraint decoding offers strong lower-resource alternatives. Our results show that small LLMs can match or even surpass larger baselines, with our best configuration based on Qwen3-1.7B achieving an average score +9.2 points higher than Qwen3-32B. We release a comprehensive collection of all the publicly available Italian medical datasets for NLP tasks, together with our top-performing models. Furthermore, we release an Italian dataset of 126M words from the Emergency Department of an Italian Hospital, and 175M words from various sources that we used for continual pre-training.