mlsanityData to 7 Oct 2026

GEAR-Seg: A Grounded Explainable Agent for Reasoning Segmentation and Data Engine

arXiv 2607.00544 ↗First posted 1 Jul 2026cs.CVcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Reasoning segmentation requires localizing targets based on complex, implicit queries. Current end-to-end models typically entangle perception and deduction into an opaque black box, severely limiting interpretability and scalability. To address this, we propose GEAR-Seg (Grounded Explainable Agent for Reasoning Segmentation), an explicitly decoupled agent that shifts the paradigm by translating visual pixels into dense, attribute-rich text. By decoupling class-agnostic segmentation, semantic description, and Large Language Model (LLM) deduction, GEAR-Seg transforms implicit reasoning into an explicit, trackable logic chain. As a zero-shot inference framework, it achieves highly competitive performance across diverse reasoning and fine-grained referring segmentation benchmarks. Furthermore, GEAR-Seg inherently functions as a highly scalable data engine. Utilizing this engine, we construct GEAR-131K, a massive benchmark (over 38k images, 656k QA-mask pairs) introducing a multifaceted taxonomy tailored for complex real-world manipulation-oriented reasoning. Finally, distillation experiments demonstrate that lightweight models supervised exclusively by our automated pipeline closely match the upper-bound performance of costly human-annotated baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 39 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

30 of 39 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation1 Apr
  2. 2025MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentation18 Mar
  3. 2024LLM-Seg: Bridging Image Segmentation and Large Language Model Reasoning12 Apr
  4. 2023PixelLM: Pixel Reasoning with Large Multimodal Model4 Dec
  5. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug

Other references · 25

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2023Segment Anythingcited by 8,158
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  7. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  8. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  9. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  10. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  11. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  12. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
13 more
  1. 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 454
  2. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  3. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  4. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  5. 2016Segmentation from Natural Language Expressionscited by 257
  6. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  7. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  8. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
  9. 2020Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmaxcited by 152
  10. 2024SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentationcited by 68
  11. 2022Infinite Recommendation Networks: A Data-Centric Approachcited by 20
  12. 2020Neural Network Compression Framework for fast model inferencecited by 18
  13. 2024Learn from Foundation Model: Fruit Detection Model without Manual Annotationcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Seg-Agent: Test-Time Multimodal Reasoning for Training-Free Language-Guided Segmentation
  2. 2026CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction
  3. 2026DR2Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Models
  4. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement
  5. 2026CR-Seg: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation
  6. 2026AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation
  7. 2026SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation
  8. 2024CoReS: Orchestrating the Dance of Reasoning and Segmentation