# MediSee: Reasoning-based Pixel-level Perception in Medical Images

arXiv: https://arxiv.org/abs/2504.11008 · HTML: https://mlsanity.com/paper/2504.11008/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Builds on (most cited first, 24 of 24)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [GLaMM: Pixel Grounding Large Multimodal Model](https://mlsanity.com/paper/2311.03356/) (2023)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [Medical Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2009.13120/) (2020)
- [The Importance of Skip Connections in Biomedical Image Segmentation](https://mlsanity.com/paper/1608.04117/) (2016)
- [The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT](https://mlsanity.com/paper/2307.01984/) (2023)
- [SegVol: Universal and Interactive Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2311.13385/) (2023)
- [DeepCut: Object Segmentation from Bounding Box Annotations using Convolutional Neural Networks](https://mlsanity.com/paper/1605.07866/) (2016)
- [SA-Med2D-20M Dataset: Segment Anything in 2D Medical Imaging with 20 Million masks](https://mlsanity.com/paper/2311.11969/) (2023)
- [LLM-Seg: Bridging Image Segmentation and Large Language Model Reasoning](https://mlsanity.com/paper/2404.08767/) (2024)
- [ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image](https://mlsanity.com/paper/2312.07381/) (2023)
- [Interactive Medical Image Segmentation: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2411.12814/) (2024)
- [Intracranial Hemorrhage Segmentation Using Deep Convolutional Model](https://mlsanity.com/paper/1910.08643/) (2019)
- [MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentation](https://mlsanity.com/paper/2503.13881/) (2025)
- [Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE](https://mlsanity.com/paper/2409.17508/) (2024)
- [Organ at Risk Segmentation in Head and Neck CT Images by Using a Two-Stage Segmentation Framework Based on 3D U-Net](https://mlsanity.com/paper/1809.00960/) (2018)

## Built on by (most cited first, 8 of 8)

- [MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision](https://mlsanity.com/paper/2508.08177/) (2025)
- [IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation](https://mlsanity.com/paper/2601.03054/) (2026)
- [SAMPO-Path: Segmentation Intent-Aligned Preference Optimization for Pathology Foundation Model Segmentation](https://mlsanity.com/paper/2508.02464/) (2025)
- [Sim4Seg: Boosting Multimodal Multi-disease Medical Diagnosis Segmentation with Region-Aware Vision-Language Similarity Masks](https://mlsanity.com/paper/2511.06665/) (2025)
- [MediRound: Multi-Round Entity-Level Reasoning Segmentation in Medical Images](https://mlsanity.com/paper/2511.12110/) (2025)
- [Unlocking the Forgery Detection Potential of Vanilla MLLMs: A Novel Training-Free Pipeline](https://mlsanity.com/paper/2511.13442/) (2025)
- [MedVeriSeg: Teaching LISA-Like Medical Segmentation Models to Verify Query Validity Without Extra Training](https://mlsanity.com/paper/2604.10242/) (2026)
- [MedUP: Awakening Unified Understanding and Perception in Medical Vision-Language Models](https://mlsanity.com/paper/2608.10635/) (2026)

## Abstract (arXiv)

Despite remarkable advancements in pixel-level medical image perception, existing methods are either limited to specific tasks or heavily rely on accurate bounding boxes or text labels as input prompts. However, the medical knowledge required for input is a huge obstacle for general public, which greatly reduces the universality of these methods. Compared with these domain-specialized auxiliary information, general users tend to rely on oral queries that require logical reasoning. In this paper, we introduce a novel medical vision task: Medical Reasoning Segmentation and Detection (MedSD), which aims to comprehend implicit queries about medical images and generate the corresponding segmentation mask and bounding box for the target object. To accomplish this task, we first introduce a Multi-perspective, Logic-driven Medical Reasoning Segmentation and Detection (MLMR-SD) dataset, which encompasses a substantial collection of medical entity targets along with their corresponding reasoning. Furthermore, we propose MediSee, an effective baseline model designed for medical reasoning segmentation and detection. The experimental results indicate that the proposed method can effectively address MedSD with implicit colloquial queries and outperform traditional medical referring segmentation methods.