Footprint-Guided Exemplar-Free Continual Histopathology Report Generation
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Rapid progress in vision-language modeling has enabled pathology report generation from gigapixel whole-slide images, but most approaches assume static training with simultaneous access to all data. In clinical deployment, however, new organs, institutions, and reporting conventions emerge over time, and sequential fine-tuning can cause catastrophic forgetting. We introduce an exemplar-free continual learning framework for WSI-to-report generation that avoids storing raw slides or patch exemplars. The core idea is a compact domain footprint built in a frozen patch-embedding space: a small codebook of representative morphology tokens together with slide-level co-occurrence summaries and lightweight patch-count priors. These footprints support generative replay by synthesizing pseudo-WSI representations that reflect domain-specific morphological mixtures, while a teacher snapshot provides pseudo-reports to supervise the updated model without retaining past data. To address shifting reporting conventions, we distill domain-specific linguistic characteristics into a compact style descriptor and use it to steer generation. At inference, the model identifies the most compatible descriptor directly from the slide signal, enabling domain-agnostic setup without requiring explicit domain identifiers. Evaluated across multiple public continual learning benchmarks, our approach outperforms exemplar-free and limited-buffer rehearsal baselines, highlighting footprint-based generative replay as a practical solution for deployment in evolving clinical settings.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 23 references that are in the map. First posted on arXiv 27 Feb 2026.
Builds on
18 of 23 references.
References · 18
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2016Learning without Forgettingcited by 2,465
- 2017Gradient Episodic Memory for Continual Learningcited by 1,871
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
- 2018Experience Replay for Continual Learningcited by 1,074
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
- 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
- 2023Progressive Prompts: Continual Learning for Language Modelscited by 195
- 2018Don't forget, there is more than forgetting: new metrics for Continual Learningcited by 113
- 2019Latent Replay for Real-Time Continual Learningcited by 97
- 2023WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Imagescited by 31
6 more
- 2023Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospectscited by 22
- 2024Continual Domain Incremental Learning for Privacy-aware Digital Pathologycited by 5
- 2024Characterizing Continual Learning Scenarios and Strategies for Audio Analysiscited by 3
- 2025Attention-based Generative Latent Replay: A Continual Learning Approach for WSI Analysiscited by 2
- 2025Modality-Agnostic Brain Lesion Segmentation with Privacy-aware Continual Learningcited by 1
- 2026Towards Modality-Agnostic Continual Domain-Incremental Brain Lesion Segmentationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024PathAlign: A vision-language model for whole slide images in histopathology
- 2025Histopathology Image Report Generation by Vision Language Model with Multimodal In-Context Learning
- 2026Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
- 2025MPath: Multimodal Pathology Report Generation from Whole Slide Images
- 2025PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation
- 2026Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation
- 2026Decomposing Whole Slide Image Report Generation with Graph-Constrained Multiple Instance Learning Workflows
- 2026Semantic Context-aware mOdality fUsion Transformer (SCOUT): A Context-Aware Multimodal Transformer for Concept-Grounded Pathology Report Generation