mlsanityData to 14 Oct 2026

TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records

arXiv 2503.04176 ↗First posted 6 Mar 2025cs.AI cs.CE cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have emerged as promising tools for assisting in medical tasks, yet processing Electronic Health Records (EHRs) presents unique challenges due to their longitudinal nature. While LLMs' capabilities to perform medical tasks continue to improve, their ability to reason over temporal dependencies across multiple patient visits and time frames remains unexplored. We introduce TIMER (Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records), a framework that incorporate instruction-response pairs grounding to different parts of a patient's record as a critical dimension in both instruction evaluation and tuning for longitudinal clinical records. We develop TIMER-Bench, the first time-aware benchmark that evaluates temporal reasoning capabilities over longitudinal EHRs, as well as TIMER-Instruct, an instruction-tuning methodology for LLMs to learn reasoning over time. We demonstrate that models fine-tuned with TIMER-Instruct improve performance by 7.3% on human-generated benchmarks and 9.2% on TIMER-Bench, indicating that temporal instruction-tuning improves model performance for reasoning over EHR.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 27 of its 60 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

27 of 60 references.

References · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  5. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  6. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  7. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  8. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 417
  9. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
  10. 2024Long-context LLMs Struggle with Long In-context Learningcited by 286
  11. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  12. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
15 more
  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  2. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 215
  3. 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 99
  4. 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 90
  5. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
  6. 2023MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Recordscited by 56
  7. 2024Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating Worldcited by 52
  8. 2024Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRscited by 39
  9. 2023Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Datacited by 32
  10. 2023BioInstruct: Instruction Tuning of Large Language Models for Biomedical Natural Language Processingcited by 25
  11. 2024Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Datacited by 22
  12. 2024Biomedical Visual Instruction Tuning with Clinician Preference Alignmentcited by 21
  13. 2023Exploring the Effectiveness of Instruction Tuning in Biomedical Language Processingcited by 16
  14. 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Timecited by 13
  15. 2024Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Datacited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 26.

34 papers.

Sort:

By area:

  1. 2025Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learningcited by 14
  2. 2025Reasoning LLMs in the Medical Domain: A Literature Surveycited by 6
  3. 2025Building the EHR Foundation Model via Next Event Predictioncited by 6
  4. 2026One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Modelscited by 5
  5. 2025Hypercube-Based Retrieval-Augmented Generation for Scientific Question-Answeringcited by 4
  6. 2026AgentEHR: Advancing Autonomous Clinical Decision-Making via Retrospective Summarizationcited by 4
  7. 2026RubricsTree: Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skillscited by 4
  8. 2026CLIR-Bench: Benchmarking Multimodal Question Answering over Irregular Clinical Time Seriescited by 4
  9. 2025PMOA-TTS: Introducing the PubMed Open Access Textual Times Series Corpuscited by 3
  10. 2025Traj-CoA: Patient Trajectory Modeling via Chain-of-Agents for Lung Cancer Risk Predictioncited by 2
  11. 2026Large Language Models Lack Temporal Awareness of Medical Knowledgecited by 2
  12. 2026EHRBench: An Automated and Reliable EHR-based Benchmark for Clinical Decision Making with LLMscited by 2
22 more
  1. 2026PORTER: Language-Grounded Event Representations for Portable Structured EHR Foundation Modelscited by 2
  2. 2025CancerGUIDE: Cancer Guideline Understanding via Internal Disagreement Estimationcited by 1
  3. 2026Agentic clinical reasoning over longitudinal myeloma records: a retrospective evaluation against expert consensuscited by 1
  4. 2026ObGynLongBench: Revealing the Evidence-to-EHR Gap in Longitudinal EHR Decision-Makingcited by 1
  5. 2025Early Risk Prediction with Temporally and Contextually Grounded Clinical Language Processingcited by 0
  6. 2026Dense Feature Learning via Linear Structure Preservation in Medical Datacited by 0
  7. 2026ESL-Bench: An Event-Driven Synthetic Longitudinal Benchmark for Health Agentscited by 0
  8. 2026LiFT: How to Enable In-Context Learning for Longitudinal Modellingcited by 0
  9. 2026Traj-Evolve: A Self-Evolving Multi-Agent System for Patient Trajectory Modeling in Lung Cancer Early Detectioncited by 0
  10. 2026VISTA Architect: A graph database-oriented health AI system demonstrated in multidisciplinary tumor boardscited by 0
  11. 2026Chronofy: A Temporal-Logical Decay Architecture for Information Validity in Time-Aware Retrieval-Augmented Generationcited by 0
  12. 2026PredAct-Bench: Benchmarking Tool-Augmented Dialogue under Controlled Tool Noisecited by 0
  13. 2026MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agentscited by 0
  14. 2026MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domaincited by 0
  15. 2026Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudicationcited by 0
  16. 2026Grounded Adjudication of Variations across Extracted TimeLines (GAVEL): Comparing Clinical Timelines Against Their Case Reportscited by 0
  17. 2026CliniCIRCA: A Modular LLM Framework for Constructing Longitudinal Mental Health Patient Journeys from Raw EHR Narrativescited by 0
  18. 2026Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmarkcited by 0
  19. 2026SMARtCARE: Privacy-Preserving Agentic AI Systems for Bounded-Autonomy Clinical Decision Supportcited by 0
  20. 2026Personalized State-Transition-Aware Memory for Clinical Agentscited by 0
  21. 2026A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combinedcited by 0
  22. 2026Large language models exhibit unreliable updating of clinical judgment as patient evidence evolvescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction
  2. 2026Can LLMs Reason Over Long Horizons? An Empirical Evaluation of Context Strategies for Longitudinal Clinical Reasoning
  3. 2026Beyond Prediction: Longitudinal Reasoning in EHR-Integrated Clinical AI ↗
  4. 2025Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs
  5. 2025O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning
  6. 2026Medical Reasoning with Large Language Models: A Survey and MR-Bench
  7. 2025EHRStruct: A Comprehensive Benchmark Framework for Evaluating Large Language Models on Structured Electronic Health Record Tasks
  8. 2026EHRWorld: A Patient-Centric Medical World Model for Long-Horizon Clinical Trajectories