# Performance of Large Language Models in Answering Critical Care Medicine Questions

arXiv: https://arxiv.org/abs/2509.19344 · HTML: https://mlsanity.com/paper/2509.19344/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 3 references that are in the map.
- First posted on arXiv 16 Sep 2025.

## Builds on (most cited first, 1 of 1)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)

## Abstract (arXiv)

Large Language Models have been tested on medical student-level questions, but their performance in specialized fields like Critical Care Medicine (CCM) is less explored. This study evaluated Meta-Llama 3.1 models (8B and 70B parameters) on 871 CCM questions. Llama3.1:70B outperformed 8B by 30%, with 60% average accuracy. Performance varied across domains, highest in Research (68.4%) and lowest in Renal (47.9%), highlighting the need for broader future work to improve models across various subspecialty domains.