mlsanityData to 14 Oct 2026

Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology

arXiv 2605.18419 ↗First posted 18 May 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-language models (VLMs) can couple visual perception with open-ended clinical reasoning, making them attractive for computational histopathology. However, fine-tuning billions of parameters on scarce, expert-annotated pathology data is prohibitive, while in-context learning (ICL), which conditions the VLM on demonstrative image-text pairs without parameter updates, suffers from high sensitivity to which examples are selected and how the query is phrased, producing unreliable diagnostics. Existing selection strategies rely on query-dependent nearest-neighbour retrieval that ignores global data structure, require costly parameter updates, or disregard the joint vision-text embedding geometry of VLMs. We propose GAUC, a training-free coreset selection method operating directly in the pre-trained multimodal embedding space. GAUC jointly optimises three objectives: (1) a Maximum Mean Discrepancy term enforcing distributional fidelity between coreset and full dataset, (2) an Effective Mutual Information Difference regulariser bounding performance degradation under prompt paraphrases by exploiting the VLM's joint vision-text alignment, and (3) a predictive-uncertainty (entropy) penalty suppressing ambivalent, hallucination-prone outputs. On CRC-100K and MHIST across multiple open-source VLM architectures, GAUC matches the accuracy of the strongest ICL selection and dataset-distillation baselines while substantially improving calibration, prompt robustness, and hallucination rates, all without a single gradient update.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 32 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

22 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024In-context learning enables multimodal large language models to classify cancer pathology images12 Mar
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  5. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565
  6. 2022Dataset Distillation by Matching Training Trajectoriescited by 442
  7. 2021Dataset Condensation with Distribution Matchingcited by 365
  8. 2024What matters when building vision-language models?cited by 326
  9. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  10. 2021A Petri Dish for Histopathology Image Analysiscited by 64
  11. 2024Isambard-AI: a leadership class supercomputer optimised specifically for Artificial Intelligencecited by 62
  12. 2023How to Configure Good In-Context Sequence for Visual Question Answeringcited by 33
8 more
  1. 2025Taming Diffusion for Dataset Distillation with High Representativenesscited by 18
  2. 2025Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approachcited by 16
  3. 2025Mimic In-Context Learning for Multimodal Taskscited by 12
  4. 2023Realistic Data Enrichment for Robust Image Segmentation in Histopathologycited by 7
  5. 2023Understanding Silent Failures in Medical Image Classificationcited by 5
  6. 2024URCDM: Ultra-Resolution Image Synthesis in Histopathologycited by 4
  7. 2025Efficient and Effective In-context Demonstration Selection with Coresetcited by 2
  8. 2025Toward More Reliable Artificial Intelligence: Reducing Hallucinations in Vision-Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026TraceICL: Thread-Resolved Demonstration Curation with Consistency Calibration for Robust Multimodal In-Context Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design
  2. 2024Boosting Vision-Language Models for Histopathology Classification: Predict all at once
  3. 2025Effortless Vision-Language Model Specialization in Histopathology without Annotation
  4. 2025Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathology
  5. 2025Histopathology Image Report Generation by Vision Language Model with Multimodal In-Context Learning
  6. 2026Histopath-C: Towards Realistic Domain Shifts for Histopathology Vision-Language Adaptation
  7. 2025Leveraging Vision-Language Embeddings for Zero-Shot Learning in Histopathology Images
  8. 2024PathAlign: A vision-language model for whole slide images in histopathology