# Contextual Evaluation of Large Language Models for Classifying Tropical and Infectious Diseases

arXiv: https://arxiv.org/abs/2409.09201 · HTML: https://mlsanity.com/paper/2409.09201/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 5 of 5)

- [Capabilities of Gemini Models in Medicine](https://mlsanity.com/paper/2404.18416/) (2024)
- [Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health](https://mlsanity.com/paper/2306.10070/) (2023)
- [A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models](https://mlsanity.com/paper/2403.12025/) (2024)
- [The Case for Globalizing Fairness: A Mixed Methods Study on Colonialism, AI, and Health in Africa](https://mlsanity.com/paper/2403.03357/) (2024)
- [Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study](https://mlsanity.com/paper/2401.09637/) (2024)

## Built on by (most cited first, 1 of 1)

- [Healthy LLMs? Benchmarking LLM Knowledge of UK Government Public Health Information](https://mlsanity.com/paper/2505.06046/) (2025)

## Abstract (arXiv)

While large language models (LLMs) have shown promise for medical question answering, there is limited work focused on tropical and infectious disease-specific exploration. We build on an opensource tropical and infectious diseases (TRINDs) dataset, expanding it to include demographic and semantic clinical and consumer augmentations yielding 11000+ prompts. We evaluate LLM performance on these, comparing generalist and medical LLMs, as well as LLM outcomes to human experts. We demonstrate through systematic experimentation, the benefit of contextual information such as demographics, location, gender, risk factors for optimal LLM response. Finally we develop a prototype of TRINDs-LM, a research tool that provides a playground to navigate how context impacts LLM outputs for health.