# ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs

arXiv: https://arxiv.org/abs/2609.31448 · HTML: https://mlsanity.com/paper/2609.31448/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 18 of 18)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization](https://mlsanity.com/paper/2309.07430/) (2023)
- [Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement](https://mlsanity.com/paper/2503.01875/) (2025)
- [ConCare: Personalized Clinical Feature Embedding via Capturing the Healthcare Context](https://mlsanity.com/paper/1911.12216/) (2019)
- [Scaling Laws for Forgetting When Fine-Tuning Large Language Models](https://mlsanity.com/paper/2401.05605/) (2024)
- [ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset](https://mlsanity.com/paper/2506.20093/) (2025)
- [ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaboration](https://mlsanity.com/paper/2410.02551/) (2024)
- [CaTS-Bench: Can Language Models Describe Time Series?](https://mlsanity.com/paper/2509.20823/) (2025)
- [VITA: 'Carefully Chosen and Weighted Less' Is Better in Medication Recommendation](https://mlsanity.com/paper/2312.12100/) (2023)
- [ClinicRealm: Re-evaluating Large Language Models with Conventional Machine Learning for Non-Generative Clinical Prediction Tasks](https://mlsanity.com/paper/2407.18525/) (2024)
- [From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?](https://mlsanity.com/paper/2505.24030/) (2025)
- [MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis](https://mlsanity.com/paper/2510.07513/) (2025)
- [CLIR-Bench: Benchmarking Multimodal Question Answering over Irregular Clinical Time Series](https://mlsanity.com/paper/2607.09880/) (2026)
- [Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and Obstetrics](https://mlsanity.com/paper/2602.00726/) (2026)

## Abstract (arXiv)

Clinical prediction models estimate risk from patient measurements, while large language models support medical text understanding and question answering. Yet their language capabilities do not ensure accurate prediction from structured, high-dimensional clinical time series. Improving this ability would connect risk estimation with flexible questions about a patient's evolving condition. We introduce ViSTA, a compact adapter that incorporates irregular numerical measurements into a pretrained vision-language model's chart representations. It learns corrections to visual tokens while leaving all pretrained parameters unchanged. On MIMIC-IV, ViSTA has the highest mean scores among the compared adaptations on all four metrics for acute kidney injury and mortality prediction across models with 2-9 billion parameters. With 0.516 million trainable parameters, the 2-billion-parameter model reaches an area under the ROC curve of 0.7376 for acute kidney injury, compared with GPT-5.6 Sol's 0.7380 with text input and high reasoning effort. Training for temporal question answering yields 69.27% accuracy at 4 billion parameters with over 90% fewer trainable parameters than low-rank adaptation using charts or numerical text, at a 2.82-4.88 percentage-point accuracy gap. ViSTA extends pretrained language models to numerical prediction and temporal questions.