mlsanityData to 14 Oct 2026

Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model

arXiv 2505.19242 ↗First posted 25 May 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Image segmentation is a fundamental task in computer vision, aimed at partitioning an image into semantically meaningful regions. Referring image segmentation extends this task by using natural language expressions to localize specific objects, requiring effective integration of visual and linguistic information. In this work, we propose SegVLM, a vision-language model that incorporates architectural improvements to enhance segmentation accuracy and cross-modal alignment. The model integrates squeeze-and-excitation (SE) blocks for dynamic feature recalibration, deformable convolutions for geometric adaptability, and residual connections for deep feature learning. We also introduce a novel referring-aware fusion (RAF) loss that balances region-level alignment, boundary precision, and class imbalance. Extensive experiments and ablation studies demonstrate that each component contributes to consistent performance improvements. SegVLM also shows strong generalization across diverse datasets and referring expression scenarios.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 75 references that are in the map. First posted on arXiv 25 May 2025.

Builds on

53 of 75 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentation26 Feb
  2. 2023Voila-A: Aligning Vision-Language Models with User's Gaze Attention22 Dec
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan
  5. 2021UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling23 Nov

Other references · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2017Mask R-CNNcited by 7,277
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  12. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
36 more
  1. 2017Deformable Convolutional Networkscited by 1,488
  2. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  3. 2016Modeling Context in Referring Expressionscited by 1,063
  4. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  6. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  7. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  8. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  9. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  10. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  11. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  12. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  13. 2018Context Encoding for Semantic Segmentationcited by 290
  14. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  15. 2016Segmentation from Natural Language Expressionscited by 257
  16. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  17. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  18. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  19. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  20. 2022OmniVL:One Foundation Model for Image-Language and Video-Language Taskscited by 143
  21. 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
  22. 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
  23. 2018Dynamic Multimodal Instance Segmentation guided by natural language queriescited by 83
  24. 2018Interactive Visual Grounding of Referring Expressions for Human-Robot Interactioncited by 56
  25. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  26. 2023Referring Image Segmentation Using Text Supervisioncited by 35
  27. 2024OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modelingcited by 28
  28. 2022Fast-MoCo: Boost Momentum-based Contrastive Learning with Combinatorial Patchescited by 16
  29. 2021MaIL: A Unified Mask-Image-Language Trimodal Network for Referring Image Segmentationcited by 14
  30. 2023MMSFormer: Multimodal Transformer for Material and Semantic Segmentationcited by 12
  31. 2023Relational Contrastive Learning for Scene Text Recognitioncited by 6
  32. 2017Grounding Spatio-Semantic Referring Expressions for Human-Robot Interactioncited by 5
  33. 2022Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learningcited by 4
  34. 2023MMNet: Multi-Mask Network for Referring Image Segmentationcited by 2
  35. 2023RISAM: Referring Image Segmentation via Mutual-Aware Attention Featurescited by 2
  36. 2024Fuse & Calibrate: A bi-directional Vision-Language Guided Framework for Referring Image Segmentationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Spatio-Semantic Expert Routing Architecture with Mixture-of-Experts for Referring Image Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GenSeg-R1: RL-Driven Vision-Language Grounding for Fine-Grained Referring Segmentation
  2. 2026Qwen3-VL-Seg: Unlocking Open-World Referring Segmentation with Vision-Language Grounding
  3. 2025LGD: Leveraging Generative Descriptions for Zero-Shot Referring Image Segmentation
  4. 2024InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Models
  5. 2026An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentation
  6. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model
  7. 2025Unlocking the Potential of MLLMs in Referring Expression Segmentation via a Light-weight Mask Decoder
  8. 2026ResAgent: Entropy-based Prior Point Discovery and Visual Reasoning for Referring Expression Segmentation