# Conversational Image Segmentation: Grounding Abstract Concepts with Scalable Supervision

arXiv: https://arxiv.org/abs/2602.13195 · HTML: https://mlsanity.com/paper/2602.13195/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023)
- [GLaMM: Pixel Grounding Large Multimodal Model](https://mlsanity.com/paper/2311.03356/) (2023)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [Segmentation from Natural Language Expressions](https://mlsanity.com/paper/1603.06180/) (2016)
- [Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement](https://mlsanity.com/paper/2503.06520/) (2025)
- [PixelLM: Pixel Reasoning with Large Multimodal Model](https://mlsanity.com/paper/2312.02228/) (2023)
- [Fake it till you make it: Learning transferable representations from synthetic ImageNet clones](https://mlsanity.com/paper/2212.08420/) (2022)
- [PACO: Parts and Attributes of Common Objects](https://mlsanity.com/paper/2301.01795/) (2023)
- [ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models](https://mlsanity.com/paper/2204.08790/) (2022)
- [Recurrent Multimodal Interaction for Referring Image Segmentation](https://mlsanity.com/paper/1703.07939/) (2017)
- [xGen-MM (BLIP-3): A Family of Open Large Multimodal Models](https://mlsanity.com/paper/2408.08872/) (2024)
- [ReSTR: Convolution-free Referring Image Segmentation Using Transformers](https://mlsanity.com/paper/2203.16768/) (2022)
- [PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding](https://mlsanity.com/paper/2504.13180/) (2025)
- [EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model](https://mlsanity.com/paper/2406.20076/) (2024)
- [Universal Segmentation at Arbitrary Granularity with Language Instruction](https://mlsanity.com/paper/2312.01623/) (2023)
- [COCONut: Modernizing COCO Segmentation](https://mlsanity.com/paper/2404.08639/) (2024)
- [COCONut-PanCap: Joint Panoptic Segmentation and Grounded Captions for Fine-Grained Understanding and Generation](https://mlsanity.com/paper/2502.02589/) (2025)

## Built on by (most cited first, 3 of 3)

- [Vision as Unified Multimodal Generation](https://mlsanity.com/paper/2607.06560/) (2026)
- [Affordance Agent Harness: Verification-Gated Skill Orchestration](https://mlsanity.com/paper/2605.00663/) (2026)
- [VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence](https://mlsanity.com/paper/2605.20676/) (2026)

## Abstract (arXiv)

Conversational image segmentation grounds abstract, intent-driven concepts into pixel-accurate masks. Prior work on referring image grounding focuses on categorical and spatial queries (e.g., "left-most apple") and overlooks functional and physical reasoning (e.g., "where can I safely store the knife?"). We address this gap and introduce Conversational Image Segmentation (CIS) and ConverSeg, a benchmark spanning entities, spatial relations, intent, affordances, functions, safety, and physical reasoning. We also present ConverSeg-Net, which fuses strong segmentation priors with language understanding, and an AI-powered data engine that generates prompt-mask pairs without human supervision. We show that current language-guided segmentation models are inadequate for CIS, while ConverSeg-Net trained on our data engine achieves significant gains on ConverSeg and maintains strong performance on existing language-guided segmentation benchmarks. Project webpage: https://glab-caltech.github.io/converseg/