Enhancing Brain MRI Anomaly Detection and Reasoning with ROI Rethink and Synthetic Data
On the map: LLM reasoning → Clinical and biomedical applications
also in 2 more
Abstract (arXiv)
Medical vision-language models typically generate diagnoses through single-pass inference without indicating which image regions support their conclusions. This lack of spatial grounding limits clinical utility: outputs cannot be audited, and models may hallucinate findings on normal scans. We present BrReMark (Brain Rethink via ROI Marking), a framework that introduces explicit region marking into brain MRI diagnosis. The model first generates hypotheses about potential abnormalities and grounds them through explicit bounding box marking, then verifies conclusions by re-examining the marked evidence. Training combines supervised fine-tuning on structured reasoning trajectories with reinforcement learning using a composite reward over localization accuracy and diagnostic reasoning. Furthermore, we integrate a domain randomization-based pathology synthesis augmentation strategy to improve the model's generalizability to out-of-distribution (OOD) data. On internal benchmark, BrReMark improves mAP50 from 0.74% to 37.54% compared to the base model, while achieving 21.57% Clinical F1 and 45.26% diagnostic accuracy. On NOVA OOD benchmark, it also achieves competitive overall performance with a 45.7% reduction in false positives compared to the state-of-the-art, indicating reduced hallucination on rare pathologies. These findings suggest that explicit hypothesis-verification grounding is a practical path toward trustworthy open-ended brain MRI diagnosis across both in-distribution and OOD settings.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 19 of its 27 references that are in the map. First posted on arXiv 24 Jun 2026.
Builds on
19 of 27 references.
Closest ancestors · 2
Same direction, newest first
- 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
- 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning26 Feb
Other references · 17
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
- 2021The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classificationcited by 518
- 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 410
- 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 237
5 more
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learningcited by 163
- 2025GRIT: Teaching MLLMs to Think with Imagescited by 125
- 2022ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation datasetcited by 106
- 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
- 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025NOVA: A Benchmark for Anomaly Localization and Clinical Reasoning in Brain MRI
- 2025Benchmark Success, Clinical Failure: When Reinforcement Learning Optimizes for Benchmarks, Not Patients
- 2026Confident but Unreliable: A Behavioral Safety Audit of Vision-Language Models on Brain MRI
- 2025An Explainable Diagnostic Framework for Neurodegenerative Dementias via Reinforcement-Optimized LLM Reasoning
- 2025MedReason-R1: Learning to Reason for CT Diagnosis with Reinforcement Learning and Local Zoom
- 2026CoRE: Concept-Reasoning Expansion for Continual Brain Lesion Segmentation
- 2026GAZE: Grounded Agentic Zero-shot Evaluation with Viewer-Level Tools and Literature Retrieval on Rare Brain MRI
- 2025MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosis