# Are Clinical T5 Models Better for Clinical Text?

arXiv: https://arxiv.org/abs/2412.05845 · HTML: https://mlsanity.com/paper/2412.05845/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 64 references that are in the map.
- First posted on arXiv 8 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains](https://mlsanity.com/paper/2402.10373/) (2024)
- [SciFive: a text-to-text transformer model for biomedical literature](https://mlsanity.com/paper/2106.03598/) (2021)
- [Do We Still Need Clinical Language Models?](https://mlsanity.com/paper/2302.08091/) (2023)
- [Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding](https://mlsanity.com/paper/2305.12031/) (2023)
- [Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain](https://mlsanity.com/paper/2307.03042/) (2023)
- [MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answering](https://mlsanity.com/paper/2406.06573/) (2024)
- [CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes](https://mlsanity.com/paper/2106.02524/) (2021)
- [Do Text-to-Text Multi-Task Learners Suffer from Task Conflict?](https://mlsanity.com/paper/2212.06645/) (2022)
- [Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium](https://mlsanity.com/paper/2403.01628/) (2024)

## Built on by (most cited first, 3 of 3)

- [FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modes](https://mlsanity.com/paper/2506.03278/) (2025)
- [Susceptibility of Large Language Models to User-Driven Factors in Medical Queries](https://mlsanity.com/paper/2503.22746/) (2025)
- [Diagnosing our datasets: How does my language model learn clinical information?](https://mlsanity.com/paper/2505.15024/) (2025)

## Abstract (arXiv)

Large language models with a transformer-based encoder/decoder architecture, such as T5, have become standard platforms for supervised tasks. To bring these technologies to the clinical domain, recent work has trained new or adapted existing models to clinical data. However, the evaluation of these clinical T5 models and comparison to other models has been limited. Are the clinical T5 models better choices than FLAN-tuned generic T5 models? Do they generalize better to new clinical domains that differ from the training sets? We comprehensively evaluate these models across several clinical tasks and domains. We find that clinical T5 models provide marginal improvements over existing models, and perform worse when evaluated on different domains. Our results inform future choices in developing clinical LLMs.