mlsanityData to 14 Oct 2026

CoReS: Orchestrating the Dance of Reasoning and Segmentation

arXiv 2404.05673 ↗First posted 8 Apr 2024cs.CVcited by 25

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

The reasoning segmentation task, which demands a nuanced comprehension of intricate queries to accurately pinpoint object regions, is attracting increasing attention. However, Multi-modal Large Language Models (MLLM) often find it difficult to accurately localize the objects described in complex reasoning contexts. We believe that the act of reasoning segmentation should mirror the cognitive stages of human visual search, where each step is a progressive refinement of thought toward the final object. Thus we introduce the Chains of Reasoning and Segmenting (CoReS) and find this top-down visual hierarchy indeed enhances the visual search process. Specifically, we propose a dual-chain structure that generates multi-modal, chain-like outputs to aid the segmentation process. Furthermore, to steer the MLLM's outputs into this intended hierarchy, we incorporate in-context inputs as guidance. Extensive experiments demonstrate the superior performance of our CoReS, which surpasses the state-of-the-art method by 6.5% on the ReasonSeg dataset. Project: https://chain-of-reasoning-and-segmentation.github.io/.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 59 of its 68 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

59 of 68 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning23 Jan
  2. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
  3. 2023DetGPT: Detect What You Need via Reasoning23 May
  4. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep

Other references · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2023Segment Anythingcited by 8,234
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  12. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
38 more
  1. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  3. 2016Pyramid Scene Parsing Networkcited by 2,479
  4. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  5. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  6. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,133
  7. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  8. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  9. 2018Dual Attention Network for Scene Segmentationcited by 1,003
  10. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  11. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  12. 2018Panoptic Segmentationcited by 802
  13. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  14. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 718
  15. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  16. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  17. 2023Segment Everything Everywhere All at Oncecited by 548
  18. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  19. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  20. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  21. 2019Panoptic-DeepLabcited by 363
  22. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  23. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  24. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
  25. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  26. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  27. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  28. 2016Modeling Context Between Objects for Referring Expression Understandingcited by 251
  29. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 232
  30. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  31. 2019Asymmetric Non-local Neural Networks for Semantic Segmentationcited by 196
  32. 2015ParseNet: Looking Wider to See Bettercited by 177
  33. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  34. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 148
  35. 2023PACO: Parts and Attributes of Common Objectscited by 146
  36. 2021Semi-supervised Semantic Segmentation with Directional Context-aware Consistencycited by 112
  37. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 111
  38. 2020Fully Convolutional Networks for Panoptic Segmentationcited by 92

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 15; 2026: 8.

25 papers.

Sort:

By area:

  1. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 28
  2. 2025UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interfacecited by 27
  3. 2025CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughnesscited by 25
  4. 2025Multimodal Referring Segmentation: A Surveycited by 22
  5. 2025HiMTok: Learning Hierarchical Mask Tokens for Image Segmentation with Large Multimodal Modelcited by 20
  6. 2025UGround: Towards Unified Visual Grounding with Unrolled Transformerscited by 14
  7. 2025Reasoning Segmentation for Images and Videos: A Surveycited by 12
  8. 2025Online Reasoning Video Segmentation with Just-in-Time Digital Twinscited by 11
  9. 2025Aligned Better, Listen Better for Audio-Visual Large Language Modelscited by 10
  10. 2025CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videoscited by 9
  11. 2025Think Before You Segment: High-Quality Reasoning Segmentation with GPT Chain of Thoughtscited by 7
  12. 2025ShowTable: Unlocking Creative Table Visualization with Collaborative Reflection and Refinementcited by 5
13 more
  1. 2024ROSE: Revolutionizing Open-Set Dense Segmentation with Patch-Wise Perceptual Large Multimodal Modelcited by 4
  2. 2025DynImg: Key Frames with Visual Prompts are Good Representation for Multi-Modal Video Understandingcited by 3
  3. 2026DR2Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Modelscited by 3
  4. 2024EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLMcited by 2
  5. 2026StAR: Segment Anything Reasonercited by 2
  6. 2026Rethinking MLLM Itself as a Segmenter with a Single Segmentation Tokencited by 2
  7. 2026AG-VAS: Anchor-Guided Zero-Shot Visual Anomaly Segmentation with Large Multimodal Modelscited by 1
  8. 2026Vision Harnessing Agent for Open Ad-hoc Segmentationcited by 1
  9. 2025Decoupling the Image Perception and Multimodal Reasoning for Reasoning Segmentation with Digital Twin Representationscited by 0
  10. 2025Fast Reasoning Segmentation for Images and Videoscited by 0
  11. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Modelscited by 0
  12. 2026DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentationcited by 0
  13. 2026Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation
  2. 2025RSVP: Reasoning Segmentation via Visual Prompting and Multi-modal Chain-of-Thought
  3. 2026Revisit to Segment: Working Memory Distillation for Reasoning Segmentation
  4. 2026CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction
  5. 2025VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning
  6. 2026CR-Seg: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation
  7. 2026Seg-Agent: Test-Time Multimodal Reasoning for Training-Free Language-Guided Segmentation
  8. 2024PRIMA: Multi-Image Vision-Language Models for Reasoning Segmentation