Navigating Gigapixel Pathology Images with Large Multimodal Models
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Recent advances in large multimodal models have allowed for the development of interactive chat models that can converse and reason about pathology whole-slide images (WSIs). However, existing slide-level chat systems are often highly specialized, typically compressing WSIs into fixed slide-level embeddings or relying on multi-component pipelines, which can lose multi-scale detail and limit generalizability beyond the target task. We present GIANT (Gigapixel Image Agent for Navigating Tissue), a simple, training-free approach that lets general-purpose multimodal models navigate WSIs on their own, iteratively selecting multi-magnification crops and aggregating evidence over time. To evaluate generalizability in WSI question answering and to promote reproducibility, we introduce MultiPathQA, a benchmark suite spanning five clinical challenges and 934 questions over 868 unique WSIs. This includes a new set of 128 pathologist-authored multiple-choice questions designed to mirror real diagnostic search and multi-scale reasoning. Using GPT-5, GIANT outperforms models specialized for pathology question answering, achieving state-of-the-art performance on four out of five benchmarks.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 45 references that are in the map. First posted on arXiv 24 Nov 2025.
Builds on
20 of 45 references.
Closest ancestors · 4
Same direction, newest first
- 2025Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasoner16 May
- 2024WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Image3 Dec
- 2024SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding15 Oct
- 2024PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathology29 Jan
Other references · 16
- 2023Large Language Modelscited by 774
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
- 2023Quilt-1M: One Million Image-Text Pairs for Histopathologycited by 192
- 2024Multimodal Whole Slide Foundation Model for Pathologycited by 156
- 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 84
- 2024PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathologycited by 80
- 2025PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathologycited by 36
- 2024WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answeringcited by 34
- 2025CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logiccited by 27
- 2025Evidence-based diagnostic reasoning with multi-agent copilot for human pathologycited by 26
- 2023Multimodal Foundation Models Exploit Text to Make Medical Image Predictionscited by 19
4 more
- 2025PathAgent: Toward Interpretable Analysis of Whole-slide Pathology Images via Large Language Model-based Agentic Reasoningcited by 18
- 2025A co-evolving agentic AI system for medical imaging analysiscited by 16
- 2025Pathology-CoT: Learning Visual Chain-of-Thought Agent from Expert Whole Slide Image Diagnosis Behaviorcited by 10
- 2025PRISM2: Unlocking Multi-Modal General Pathology AI with Clinical Dialoguecited by 8
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide imagecited by 3
- 2026PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?cited by 2
- 2026Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perceptioncited by 1
- 2026SlideBank: A Persistent Hierarchical Evidence Bank for Consistent Whole-Slide Reasoningcited by 1
- 2026PPGL-Swarm: Integrated Multimodal Risk Stratification and Hereditary Syndrome Detection in Pheochromocytoma and Paragangliomacited by 0
- 2026How Seemingly Inconsequential Design Choices Dictate Performance of LLMs in Pathologycited by 0
- 2026PathoArgus: Advancing Evidence-Grounded Long-Context Visual Reasoning across Gigapixel Whole-Slide and Multi-Slide Case Contextscited by 0
- 2026SlideBank: A Persistent Hierarchical Evidence Bank for Consistent Whole-Slide Reasoning31 Aug 2026
- 2026PathoArgus: Advancing Evidence-Grounded Long-Context Visual Reasoning across Gigapixel Whole-Slide and Multi-Slide Case Contexts18 Aug 2026
- 2026Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception9 Aug 2026
- 2026PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?30 Jul 2026
- 2026How Seemingly Inconsequential Design Choices Dictate Performance of LLMs in Pathology10 Jun 2026
- 2026PPGL-Swarm: Integrated Multimodal Risk Stratification and Hereditary Syndrome Detection in Pheochromocytoma and Paraganglioma23 Mar 2026
- 2026QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide image2 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoning
- 2026PathNavigate: A Training-Free Pathology Agent with Surprise-Guided Scan and Shared Slide Memory for Whole-Slide Image VQA
- 2025ChatEXAONEPath: An Expert-level Multimodal Large Language Model for Histopathology Using Whole Slide Images
- 2025A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model
- 2026Multimodal Model for Computational Pathology:Representation Learning and Image Compression
- 2026ReinPath: A Multimodal Reinforcement Learning Approach for Pathology
- 2025Democratising Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling
- 2026Enhancing Pathological VLMs with Cross-scale Reasoning