mlsanityData to 7 Oct 2026

Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation

arXiv 2605.30716 ↗First posted 29 May 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Generating clinically useful pathology reports for pathology cases from whole-slide images (WSIs) is challenging due to gigapixel resolution, long visual-token sequences, and the complexity of case-level reasoning, where a single case may contain multiple WSIs with heterogeneous tissues and ambiguous findings. We present a simple token-efficient vision--language model for case-level synoptic report generation that remains practical under constrained GPU memory. Our architecture follows a minimal three-component design: a frozen pathology patch encoder, a lightweight two-layer MLP vision-language aligner, and a large language model decoder, with an explicit WSI marker token to separate slides within a case. Training proceeds in two supervised stages: (1) aligner-only WSI captioning using heterogeneous WSI-text pairs, and (2) case-level supervised fine-tuning on case-report pairs for structured report generation. To reduce sequence length, we represent each slide using 512 × 512 patches at 5× magnification, which reduces the average sequence length by up to 64× times compared to the commonly used 20× patches. Combined with efficient training techniques, we enable practical training with only half a NVIDIA H100 GPU. Across both training stages, our approach achieves high ROUGE-L/METEOR/BLEU-4 scores while being substantially more efficient in memory and runtime. In AI-based evaluations, our model is consistently preferred over strong baselines. Extensive ablations characterize performance-efficiency trade-offs and identify simple choices that improve robustness in multi-WSI settings. Overall, this work provides a strong, reproducible baseline for efficient pathology report generation, lowering the barrier to multi-WSI VLM research under limited compute. Code is available at https://github.com/AtlasAnalyticsLab/PathoSynVLM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 27 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

13 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design30 Apr
  2. 2024WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Image3 Dec

Other references · 11

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  6. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  7. 2024Multimodal Whole Slide Foundation Model for Pathologycited by 156
  8. 2024PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathologycited by 80
  9. 2024HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interactioncited by 43
  10. 2023Computational Pathology: A Survey Review and The Way Forwardcited by 30
  11. 2025HISTAI: An Open-Source, Large-Scale Whole Slide Image Dataset for Computational Pathologycited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024PathAlign: A vision-language model for whole slide images in histopathology
  2. 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
  3. 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation
  4. 2024Clinical-grade Multi-Organ Pathology Report Generation for Multi-scale Whole Slide Images via a Semantically Guided Medical Text Foundation Model
  5. 2025MPath: Multimodal Pathology Report Generation from Whole Slide Images
  6. 2026HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction
  7. 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding
  8. 2026Decomposing Whole Slide Image Report Generation with Graph-Constrained Multiple Instance Learning Workflows