Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Generating clinically useful pathology reports for pathology cases from whole-slide images (WSIs) is challenging due to gigapixel resolution, long visual-token sequences, and the complexity of case-level reasoning, where a single case may contain multiple WSIs with heterogeneous tissues and ambiguous findings. We present a simple token-efficient vision--language model for case-level synoptic report generation that remains practical under constrained GPU memory. Our architecture follows a minimal three-component design: a frozen pathology patch encoder, a lightweight two-layer MLP vision-language aligner, and a large language model decoder, with an explicit WSI marker token to separate slides within a case. Training proceeds in two supervised stages: (1) aligner-only WSI captioning using heterogeneous WSI-text pairs, and (2) case-level supervised fine-tuning on case-report pairs for structured report generation. To reduce sequence length, we represent each slide using 512 × 512 patches at 5× magnification, which reduces the average sequence length by up to 64× times compared to the commonly used 20× patches. Combined with efficient training techniques, we enable practical training with only half a NVIDIA H100 GPU. Across both training stages, our approach achieves high ROUGE-L/METEOR/BLEU-4 scores while being substantially more efficient in memory and runtime. In AI-based evaluations, our model is consistently preferred over strong baselines. Extensive ablations characterize performance-efficiency trade-offs and identify simple choices that improve robustness in multi-WSI settings. Overall, this work provides a strong, reproducible baseline for efficient pathology report generation, lowering the barrier to multi-WSI VLM research under limited compute. Code is available at https://github.com/AtlasAnalyticsLab/PathoSynVLM.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 27 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
13 of 27 references.
Closest ancestors · 2
Same direction, newest first
- 2025Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design30 Apr
- 2024WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Image3 Dec
Other references · 11
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2016Gaussian Error Linear Units (GELUs)cited by 3,431
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
- 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
- 2024Multimodal Whole Slide Foundation Model for Pathologycited by 156
- 2024PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathologycited by 80
- 2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43
- 2023Computational Pathology: A Survey Review and The Way Forwardcited by 30
- 2025HISTAI: An Open-Source, Large-Scale Whole Slide Image Dataset for Computational Pathologycited by 9
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024PathAlign: A vision-language model for whole slide images in histopathology
- 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
- 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation
- 2024Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model
- 2025MPath: Multimodal Pathology Report Generation from Whole Slide Images
- 2026HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction
- 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding
- 2026Decomposing Whole Slide Image Report Generation with Graph-Constrained Multiple Instance Learning Workflows