mlsanityData to 14 Oct 2026

Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming

arXiv 2605.21652 ↗First posted 20 May 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-Language Models (VLMs) have significantly advanced medical visual question answering, yet their performance in ultrasound remains suboptimal. In clinical practice, sonographers explicitly focus on lesion regions to formulate reports, though diagnostic interpretations sometimes vary due to inherent subjectivity. However, existing VLMs are not explicitly structured to interactively zoom into lesions prior to diagnosis; moreover, they typically treat annotations as unbiased ground truths, failing to account for their inherent subjectivity and ambiguity. In this paper, we propose a framework specifically designed to consider the sonographer's cognitive workflow. We first introduce a structured Zoom-then-Diagnose paradigm, which replicates the interactive search process to enable lesion-focused reasoning. Furthermore, within the Group Relative Policy Optimization (GRPO) framework, we introduce an uncertainty-aware reward derived from stochastic group-wise rollouts to estimate prediction consistency as a proxy for model confidence. Together, these two components encourage the model to reinforce accurate predictions on clear cases while remaining cautious under ambiguity. Experiments across liver, breast, and thyroid datasets show that our framework improves lesion localization by 39.3%, demonstrating that our model has learned the ability to actively look closer and diagnose.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 26 of its 34 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

26 of 34 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025EchoVLM: Dynamic Mixture-of-Experts Vision-Language Model for Universal Ultrasound Intelligence18 Sep
  2. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
  3. 2025U2-BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding23 May
  4. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning26 Feb
  5. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun

Other references · 21

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  8. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  9. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  10. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 410
  11. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
  12. 2023Do Large Language Models Know What They Don't Know?cited by 243
9 more
  1. 2023HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledgecited by 162
  2. 2025GRIT: Teaching MLLMs to Think with Imagescited by 125
  3. 2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 75
  4. 2025Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Modelscited by 52
  5. 2023Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectivescited by 31
  6. 2025SteerConf: Steering LLMs for Confidence Elicitationcited by 20
  7. 2025Improving Medical Reasoning with Curriculum-Aware Reinforcement Learningcited by 16
  8. 2025Speckle2Self: Self-Supervised Ultrasound Speckle Reduction Without Clean Datacited by 9
  9. 2025A Chain-of-thought Reasoning Breast Ultrasound Dataset Covering All Histopathology Categoriescited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Real-World Ultrasound Understanding: Large Vision-Language Models from Multi-Image Examinations with Long-Form Reports
  2. 2026When Minor Edits Matter: LLM-Driven Prompt Attack for Medical VLM Robustness in Ultrasound
  3. 2026UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasound
  4. 2025UMind-VL: A Generalist Ultrasound Vision-Language Model for Unified Grounded Perception and Comprehensive Interpretation
  5. 2026Echo-α: Large Agentic Multimodal Reasoning Model for Ultrasound Interpretation
  6. 2025FETAL-GAUGE: A Benchmark for Assessing Vision-Language Models in Fetal Ultrasound
  7. 2025Test-Time-Scaling for Zero-Shot Diagnosis with Visual-Language Reasoning
  8. 2026ReXSonoVQA: A Video QA Benchmark for Procedure-Centric Ultrasound Understanding