mlsanityData to 14 Oct 2026

Instruction-Guided Lesion Segmentation for Chest X-rays with Automatically Generated Large-Scale Dataset

arXiv 2511.15186 ↗First posted 19 Nov 2025cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

The applicability of current lesion segmentation models for chest X-rays (CXRs) has been limited both by a small number of target labels and the reliance on complex, expert-level text inputs, creating a barrier to practical use. To address these limitations, we introduce instruction-guided lesion segmentation (ILS), a medical-domain adaptation of referring image segmentation (RIS) designed to segment diverse lesion types based on simple, user-friendly instructions. Under this task, we construct MIMIC-ILS, the first large-scale instruction-answer dataset for CXR lesion segmentation, using our fully automated multimodal pipeline that generates annotations from CXR images and their corresponding reports. MIMIC-ILS contains 1.1M instruction-answer pairs derived from 192K images and 91K unique segmentation masks, covering seven major lesion types. To empirically demonstrate its utility, we present ROSALIA, a LISA model fine-tuned on the MIMIC-ILS dataset. ROSALIA can segment diverse lesions and provide textual explanations in response to user instructions. The model achieves high accuracy in our newly proposed task, highlighting the effectiveness of our pipeline and the value of MIMIC-ILS as a foundational resource for pixel-level CXR lesion grounding. The dataset and model are available at https://github.com/checkoneee/ROSALIA.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

31 of 45 references.

References · 31

  1. 2023Segment Anythingcited by 8,234
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  4. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  5. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  6. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  7. 2022Language-driven Semantic Segmentationcited by 717
  8. 2021The Medical Segmentation Decathloncited by 628
  9. 2021Extract Free Dense Labels from CLIPcited by 583
  10. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
  11. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 509
  12. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
19 more
  1. 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 239
  2. 2023CLIP-Driven Universal Model for Organ Segmentation and Tumor Detectioncited by 224
  3. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 222
  4. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 154
  5. 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 117
  6. 2022LViT: Language meets Vision Transformer in Medical Image Segmentationcited by 111
  7. 2024Text4Seg: Reimagining Image Segmentation as Text Generationcited by 61
  8. 2022OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Imagescited by 48
  9. 2024PadChest-GR: A Bilingual Chest X-ray Dataset for Grounded Radiology Report Generationcited by 35
  10. 2023CheXmask: a large-scale dataset of anatomical segmentation masks for multi-center chest x-ray imagescited by 33
  11. 2024Interactive Medical Image Segmentation: A Benchmark Dataset and Baselinecited by 30
  12. 2023RadEdit: stress-testing biomedical vision models via diffusion image editingcited by 28
  13. 2024Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentationcited by 28
  14. 2023Accurate Fine-Grained Segmentation of Human Anatomy in Radiographs via Volumetric Pseudo-Labelingcited by 26
  15. 2022Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysiscited by 25
  16. 2023ZePT: Zero-Shot Pan-Tumor Segmentation via Query-Disentangling and Self-Promptingcited by 15
  17. 2025Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMscited by 14
  18. 2024Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignmentcited by 7
  19. 2025CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rayscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026CheXpercept: A Benchmark for Evaluating Expert-Level Lesion Perception in Chest X-rayscited by 1
  2. 2026UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasoundcited by 1
  3. 2026MedVeriSeg: Teaching LISA-Like Medical Segmentation Models to Verify Query Validity Without Extra Trainingcited by 0
  4. 2026CheXGround: Anatomical Region Tokens for Grounded Longitudinal Chest X-ray Interpretationcited by 0
  5. 2026XFlow: A Workflow Model for Instruction-Guided Lesion Segmentation in Chest X-rayscited by 0