# Supervised Fine-Tuning or In-Context Learning? Evaluating LLMs for Clinical NER

arXiv: https://arxiv.org/abs/2510.22285 · HTML: https://mlsanity.com/paper/2510.22285/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 25 Oct 2025.

## Builds on (most cited first, 7 of 7)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [Biomedical named entity recognition using BERT in the machine reading comprehension framework](https://mlsanity.com/paper/2009.01560/) (2020)
- [Extensive Evaluation of Transformer-based Architectures for Adverse Drug Events Extraction](https://mlsanity.com/paper/2306.05276/) (2023)
- [Mining Adverse Drug Reactions from Unstructured Mediums at Scale](https://mlsanity.com/paper/2201.01405/) (2022)

## Abstract (arXiv)

We study clinical Named Entity Recognition (NER) on the CADEC corpus and compare three families of approaches: (i) BERT-style encoders (BERT Base, BioClinicalBERT, RoBERTa-large), (ii) GPT-4o used with few-shot in-context learning (ICL) under simple vs. complex prompts, and (iii) GPT-4o with supervised fine-tuning (SFT). All models are evaluated on standard NER metrics over CADEC's five entity types (ADR, Drug, Disease, Symptom, Finding). RoBERTa-large and BioClinicalBERT offer limited improvements over BERT Base, showing the limit of these family of models. Among LLM settings, simple ICL outperforms a longer, instruction-heavy prompt, and SFT achieves the strongest overall performance (F1 ≈ 87.1%), albeit with higher cost. We find that the LLM achieve higher accuracy on simplified tasks, restricting classification to two labels.