# DM-Bench: Benchmarking LLMs for Personalized Decision Making in Diabetes Management

arXiv: https://arxiv.org/abs/2510.00038 · HTML: https://mlsanity.com/paper/2510.00038/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://mlsanity.com/paper/2505.08775/) (2025)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models](https://mlsanity.com/paper/2307.02028/) (2023)
- [MedCalc-Bench: Evaluating Large Language Models for Medical Calculations](https://mlsanity.com/paper/2406.12036/) (2024)
- [MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks](https://mlsanity.com/paper/2505.23802/) (2025)
- [EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summaries](https://mlsanity.com/paper/2402.16040/) (2024)
- [GlucoSynth: Generating Differentially-Private Synthetic Glucose Traces](https://mlsanity.com/paper/2303.01621/) (2023)
- [Diabetica: Adapting Large Language Model to Enhance Multiple Medical Tasks in Diabetes Care and Management](https://mlsanity.com/paper/2409.13191/) (2024)
- [Advice for Diabetes Self-Management by ChatGPT Models: Challenges and Recommendations](https://mlsanity.com/paper/2501.07931/) (2025)

## Built on by (most cited first, 2 of 2)

- [LLM-Powered Personalized Glycemic Assessment in Type 2 Diabetes with Wearable Sensor Data](https://mlsanity.com/paper/2606.12699/) (2026)
- [If Only My CGM Could Speak: A Privacy-Preserving Agent for Question Answering over Continuous Glucose Data](https://mlsanity.com/paper/2604.17133/) (2026)

## Abstract (arXiv)

We present DM-Bench, the first benchmark designed to evaluate large language model (LLM) performance across real-world decision-making tasks faced by individuals managing diabetes in their daily lives. Unlike prior health benchmarks that are either generic, clinician-facing or focused on clinical tasks (e.g., diagnosis, triage), DM-Bench introduces a comprehensive evaluation framework tailored to the unique challenges of prototyping patient-facing AI solutions in diabetes, glucose management, metabolic health and related domains. Our benchmark encompasses 7 distinct task categories, reflecting the breadth of real-world questions individuals with diabetes ask, including basic glucose interpretation, educational queries, behavioral associations, advanced decision making and long term planning. Towards this end, we compile a rich dataset comprising one month of time-series data encompassing glucose traces and metrics from continuous glucose monitors (CGMs) and behavioral logs (e.g., eating and activity patterns) from 15,000 individuals across three different diabetes populations (type 1, type 2, pre-diabetes/general health and wellness). Using this data, we generate a total of 360,600 personalized, contextual questions across the 7 tasks. We evaluate model performance on these tasks across 5 metrics: accuracy, groundedness, safety, clarity and actionability. Our analysis of 8 recent LLMs reveals substantial variability across tasks and metrics; no single model consistently outperforms others across all dimensions. By establishing this benchmark, we aim to advance the reliability, safety, effectiveness and practical utility of AI solutions in diabetes care.