GEAR-Seg: A Grounded Explainable Agent for Reasoning Segmentation and Data Engine
On the map: LLM reasoning → Multimodal reasoning
also in 1 more
Abstract (arXiv)
Reasoning segmentation requires localizing targets based on complex, implicit queries. Current end-to-end models typically entangle perception and deduction into an opaque black box, severely limiting interpretability and scalability. To address this, we propose GEAR-Seg (Grounded Explainable Agent for Reasoning Segmentation), an explicitly decoupled agent that shifts the paradigm by translating visual pixels into dense, attribute-rich text. By decoupling class-agnostic segmentation, semantic description, and Large Language Model (LLM) deduction, GEAR-Seg transforms implicit reasoning into an explicit, trackable logic chain. As a zero-shot inference framework, it achieves highly competitive performance across diverse reasoning and fine-grained referring segmentation benchmarks. Furthermore, GEAR-Seg inherently functions as a highly scalable data engine. Utilizing this engine, we construct GEAR-131K, a massive benchmark (over 38k images, 656k QA-mask pairs) introducing a multifaceted taxonomy tailored for complex real-world manipulation-oriented reasoning. Finally, distillation experiments demonstrate that lightweight models supervised exclusively by our automated pipeline closely match the upper-bound performance of costly human-annotated baselines.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 39 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
30 of 39 references.
Closest ancestors · 5
Same direction, newest first
- 2025POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation1 Apr
- 2025MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentation18 Mar
- 2024LLM-Seg: Bridging Image Segmentation and Large Language Model Reasoning12 Apr
- 2023PixelLM: Pixel Reasoning with Large Multimodal Model4 Dec
- 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
Other references · 25
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Segment Anythingcited by 8,158
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
- 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
- 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
13 more
- 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 454
- 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
- 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
- 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
- 2016Segmentation from Natural Language Expressionscited by 257
- 2023GRES: Generalized Referring Expression Segmentationcited by 252
- 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
- 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
- 2020Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmaxcited by 152
- 2024SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentationcited by 68
- 2022Infinite Recommendation Networks: A Data-Centric Approachcited by 20
- 2020Neural Network Compression Framework for fast model inferencecited by 18
- 2024Learn from Foundation Model: Fruit Detection Model without Manual Annotationcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Seg-Agent: Test-Time Multimodal Reasoning for Training-Free Language-Guided Segmentation
- 2026CoT-Seg: Rethinking Segmentation with Chain-of-Thought Reasoning and Self-Correction
- 2026DR2Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Models
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement
- 2026CR-Seg: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation
- 2026AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation
- 2026SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation
- 2024CoReS: Orchestrating the Dance of Reasoning and Segmentation