mlsanityData to 14 Oct 2026

One Patient, Many Contexts: Scaling Medical AI with Contextual Intelligence

arXiv 2506.10157 ↗First posted 11 Jun 2025cs.AI cs.CLcited by 2

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Medical AI, including clinical language models, vision-language models, and multimodal health record models, already summarizes notes, answers questions, and supports decisions. Their adaptation to new populations, specialties, or care settings often relies on fine-tuning, prompting, or retrieval from external knowledge bases. These strategies can scale poorly and risk contextual errors: outputs that appear plausible but miss critical patient or situational information. We envision context switching as a solution. Context switching adjusts model reasoning at inference without retraining. Generative models can tailor outputs to patient biology, care setting, or disease. Multimodal models can reason on notes, laboratory results, imaging, and genomics, even when some data are missing or delayed. Agent models can coordinate tools and roles based on tasks and users. In each case, context switching enables medical AI to adapt across specialties, populations, and geographies. It requires advances in data design, model architectures, and evaluation frameworks, and establishes a foundation for medical AI that scales to infinitely many contexts while remaining reliable and suited to real-world care.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 111 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

49 of 111 references.

References · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  6. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  7. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  8. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  9. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  10. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 425
  11. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
  12. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 329
37 more
  1. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  2. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
  3. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 152
  4. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 150
  5. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  6. 2025BIG-Bench Extra Hardcited by 125
  7. 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 124
  8. 2025MLGym: A New Framework and Benchmark for Advancing AI Research Agentscited by 124
  9. 20234M: Massively Multimodal Masked Modelingcited by 120
  10. 2024Empowering Biomedical Discovery with AI Agentscited by 118
  11. 2023Towards Accurate Differential Diagnosis with Large Language Modelscited by 116
  12. 2025SycEval: Evaluating LLM Sycophancycited by 108
  13. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
  14. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  15. 2024Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Expertscited by 70
  16. 2025MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agentscited by 62
  17. 2025TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Toolscited by 59
  18. 2022Multitask Vision-Language Prompt Tuningcited by 51
  19. 2024ControlAR: Controllable Image Generation with Autoregressive Modelscited by 48
  20. 2025Reasoning Language Models: A Blueprintcited by 42
  21. 2025Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learningcited by 40
  22. 2023Conformal Nucleus Samplingcited by 33
  23. 2023A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Recordscited by 31
  24. 2019Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoderscited by 27
  25. 2024Open-RAG: Enhanced Retrieval-Augmented Reasoning with Open-Source Large Language Modelscited by 25
  26. 2025Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicingcited by 25
  27. 2024OpenMedLM: Prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language modelscited by 20
  28. 2024KGARevion: An AI Agent for Knowledge-Intensive Biomedical QAcited by 20
  29. 2025How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergencecited by 15
  30. 2022RepsNet: Combining Vision with Language for Automated Medical Reportscited by 14
  31. 2024PDL: A Declarative Prompt Programming Languagecited by 14
  32. 2024Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learningcited by 13
  33. 2021Structure Inducing Pre-Trainingcited by 10
  34. 2024Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusioncited by 9
  35. 2025AutoMedPrompt: A New Framework for Optimizing LLM Medical Prompts Using Textual Gradientscited by 7
  36. 2025DoctorRAG: Medical RAG Fusing Knowledge with Patient Analogy through Textual Gradientscited by 5
  37. 2025Multimodal AI predicts clinical outcomes of drug combinations from preclinical datacited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Medical Chatbots: Meddollina and the Rise of Continuous Clinical Intelligence
  2. 2024Towards Next-Generation Medical Agent: How o1 is Reshaping Decision-Making in Medical Scenarios
  3. 2025From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine
  4. 2026Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation
  5. 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy
  6. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
  7. 2026Generated Context versus Governed State: Functional Conditions for Accountable Longitudinal Clinical Reasoning
  8. 2026MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents