# Can LLMs Reason Over Long Horizons? An Empirical Evaluation of Context Strategies for Longitudinal Clinical Reasoning

arXiv: https://arxiv.org/abs/2610.00562 · HTML: https://mlsanity.com/paper/2610.00562/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 21 of its 21 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [Gemma 4 Technical Report](https://mlsanity.com/paper/2607.02770/) (2026)
- [MedOdyssey: A Medical Domain Benchmark for Long Context Evaluation Up to 200K Tokens](https://mlsanity.com/paper/2406.15019/) (2024)
- [Evaluating Long-Term Memory for Long-Context Question Answering](https://mlsanity.com/paper/2510.23730/) (2025)
- [MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems](https://mlsanity.com/paper/2605.28732/) (2026)
- [LongMedBench: Benchmarking Medical Agents for Long-Horizon Clinical Decision-Making](https://mlsanity.com/paper/2607.09322/) (2026)
- [MedLoCoMo: A Long-Context Multi-Session Medical Dialogue Benchmark for Large Language Models](https://mlsanity.com/paper/2607.22566/) (2026)
- [Synthesis and Evaluation of Long-term History-aware Medical Dialogue](https://mlsanity.com/paper/2605.19766/) (2026)
- [AdMem: Advanced Memory for Task-solving Agents](https://mlsanity.com/paper/2606.06787/) (2026)

## Abstract (arXiv)

Longitudinal clinical reasoning requires large language models (LLMs) to identify and integrate relevant evidence distributed across extended patient histories. Although long-context models can process increasingly large amounts of information, providing more history does not necessarily make relevant evidence more accessible or improve reasoning. We compare five context strategies (Full, Recent, Episodic, Semantic, and Hybrid) on MedLoCoMo across four open-weight LLMs, examining answer correctness, robustness to query-evidence distance, and abstention on questions with unsupported premises. Episodic and Hybrid generally achieve the strongest overall accuracy, while Recent Context degrades most as supporting evidence becomes more distant; Episodic and Hybrid maintain the highest accuracy at long distances. Analysis of adversarial questions further shows that strong performance on answerable questions does not necessarily translate to successful abstention when the available history does not support the requested conclusion. These findings show that reliable longitudinal reasoning depends not only on how much history an LLM can access, but critically on how relevant evidence is selected and presented for reasoning.