mlsanityData to 14 Oct 2026

Reasoning Segmentation for Images and Videos: A Survey

arXiv 2505.18816 ↗First posted 24 May 2025cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Reasoning Segmentation (RS) aims to delineate objects based on implicit text queries, the interpretation of which requires reasoning and knowledge integration. Unlike the traditional formulation of segmentation problems that relies on fixed semantic categories or explicit prompting, RS bridges the gap between visual perception and human-like reasoning capabilities, facilitating more intuitive human-AI interaction through natural language. Our work presents the first comprehensive survey of RS for image and video processing, examining 26 state-of-the-art methods together with a review of the corresponding evaluation metrics, as well as 29 datasets and benchmarks. We also explore existing applications of RS across diverse domains and identify their potential extensions. Finally, we identify current research gaps and highlight promising future directions.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 84 of its 111 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

84 of 111 references.

References · 84

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  11. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  12. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
38 more
  1. 2024Depth Anything V2cited by 1,422
  2. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  3. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  4. 2016Modeling Context in Referring Expressionscited by 1,063
  5. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  6. 2021Transformers in Vision: A Surveycited by 1,013
  7. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  8. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  9. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  10. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  11. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  12. 2018Object Hallucination in Image Captioningcited by 669
  13. 2020Image Segmentation Using Deep Learning: A Surveycited by 660
  14. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  15. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  16. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
  17. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
  18. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  19. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  20. 2019Video Instance Segmentationcited by 393
  21. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
  22. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  23. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 232
  24. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 232
  25. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  26. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 222
  27. 2018Video Object Segmentation with Language Referring Expressionscited by 162
  28. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  29. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  30. 2020TAO: A Large-Scale Benchmark for Tracking Any Objectcited by 147
  31. 2023PACO: Parts and Attributes of Common Objectscited by 146
  32. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  33. 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
  34. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 100
  35. 2020A Survey on Instance Segmentation: State of the artcited by 85
  36. 2019Oops! Predicting Unintentional Action in Videocited by 82
  37. 2023LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Modelcited by 71
  38. 2024SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentationcited by 69

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 6.

12 papers.

  1. 2025Multimodal Referring Segmentation: A Surveycited by 22
  2. 2025Reasoning Text-to-Video Retrieval via Digital Twin Video Representations and Large Language Modelscited by 3
  3. 2025Text-Driven Reasoning Video Editing via Reinforcement Learning on Digital Twin Representationscited by 3
  4. 2026DR2Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Modelscited by 3
  5. 2026Discriminative Perception via Anchored Description for Reasoning Segmentationcited by 3
  6. 2025Temporally-Constrained Video Reasoning Segmentation and Automated Benchmark Constructioncited by 2
  7. 2025Constructing and Interpreting Digital Twin Representations for Visual Reasoning via Reinforcement Learningcited by 1
  8. 2026An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentationcited by 1
  9. 2025Fast Reasoning Segmentation for Images and Videoscited by 0
  10. 2026GTPBD-MM: A Global Terraced Parcel and Boundary Dataset with Multi-Modalitycited by 0
  11. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Modelscited by 0
  12. 2026DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation
  2. 2025Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies
  3. 2025RSVP: Reasoning Segmentation via Visual Prompting and Multi-modal Chain-of-Thought
  4. 2023LISA: Reasoning Segmentation via Large Language Model
  5. 2025Decoupling the Image Perception and Multimodal Reasoning for Reasoning Segmentation with Digital Twin Representations
  6. 2025Think Before You Segment: High-Quality Reasoning Segmentation with GPT Chain of Thoughts
  7. 2024VISA: Reasoning Video Object Segmentation via Large Language Models
  8. 2026VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation