CliniBench: A Clinical Outcome Prediction Benchmark for Generative and Encoder-Based Language Models
Not on the map. Why?
Abstract (arXiv)
With their growing capabilities, generative large language models (LLMs) are being increasingly investigated for complex medical tasks. However, their effectiveness in real-world clinical applications remains underexplored. To address this, we present CliniBench, the first benchmark that enables comparability of well-studied encoder-based classifiers and generative LLMs for discharge diagnosis prediction from admission notes in MIMIC-IV dataset. Our extensive study compares 12 generative LLMs and 3 encoder-based classifiers and demonstrates that encoder-based classifiers consistently outperform generative models in diagnosis prediction. We assess several retrieval augmentation strategies for in-context learning from similar patients and find that they provide notable performance improvements for generative LLMs.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 58 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
28 of 58 references.
References · 28
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
- 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
- 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
- 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
- 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 263
16 more
- 2018Explainable Prediction of Medical Codes from Clinical Textcited by 234
- 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 163
- 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 104
- 2023Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMscited by 62
- 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
- 2021Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integrationcited by 44
- 2023Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domaincited by 38
- 2022Validating Large Language Models with ReLMcited by 33
- 2024BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicinecited by 31
- 2024Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERTcited by 21
- 2021Literature-Augmented Clinical Outcome Predictioncited by 20
- 2023Mimic-IV-ICD: A new benchmark for eXtreme MultiLabel Classificationcited by 19
- 2023This is not correct! Negation-aware Evaluation of Language Generation Systemscited by 17
- 2023A Two-Stage Decoder for Efficient ICD Codingcited by 9
- 2022This Patient Looks Like That Patient: Prototypical Networks for Interpretable Diagnosis Prediction from Clinical Textcited by 6
- 2024Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answeringcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.