mlsanityData to 7 Oct 2026

TextRegion: Text-Aligned Region Tokens from Frozen Image-Text Models

arXiv 2505.23769 ↗First posted 29 May 2025cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Image-text models excel at image-level tasks but struggle with detailed visual understanding. While these models provide strong visual-language alignment, segmentation models like SAM2 offer precise spatial boundaries for objects. To this end, we propose TextRegion, a simple, effective, and training-free framework that combines the strengths of image-text models and SAM2 to generate powerful text-aligned region tokens. These tokens enable detailed visual understanding while preserving open-vocabulary capabilities. They can be directly applied to various downstream tasks, including open-world semantic segmentation, referring expression comprehension, and grounding. We conduct extensive evaluations and consistently achieve superior or competitive performance compared to state-of-the-art training-free methods. Additionally, our framework is compatible with many image-text models, making it highly practical and easily extensible as stronger models emerge. Code is available at: https://github.com/avaxiao/TextRegion.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 51 of its 56 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

51 of 56 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
38 more
  1. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  2. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  3. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  4. 2016Modeling Context in Referring Expressionscited by 1,054
  5. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  6. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  7. 2023Vision Transformers Need Registerscited by 844
  8. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  9. 2021Extract Free Dense Labels from CLIPcited by 580
  10. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  11. 2023Segment Everything Everywhere All at Oncecited by 546
  12. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  13. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  14. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  15. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 298
  16. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  17. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  18. 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 215
  19. 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
  20. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  21. 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 169
  22. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  23. 2022ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehensioncited by 119
  24. 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 112
  25. 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
  26. 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 110
  27. 2023A Closer Look at the Explainability of Contrastive Language-Image Pre-trainingcited by 95
  28. 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 94
  29. 2023Fine-Grained Visual Promptingcited by 81
  30. 2024Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentationcited by 61
  31. 2024In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentationcited by 38
  32. 2024Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentationcited by 38
  33. 2023Tokenize Anything via Promptingcited by 31
  34. 2024ResCLIP: Residual Attention for Training-free Dense Vision-language Inferencecited by 28
  35. 2025LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentationcited by 21
  36. 2024Region-Based Representations Revisitedcited by 18
  37. 2024Toward Interactive Regional Understanding in Vision-Large Language Modelscited by 9
  38. 2024Search and Detect: Training-Free Long Tail Object Detection via Web-Image Retrievalcited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2025RADSeg: Unleashing Parameter and Compute Efficient Zero-Shot Open-Vocabulary Segmentation Using Agglomerative Modelscited by 5
  2. 2026T-REN: Learning Text-Aligned Region Tokens Improves Dense Vision-Language Alignment and Scalabilitycited by 2
  3. 2026OV-Stitcher: A Global Context-Aware Framework for Training-Free Open-Vocabulary Semantic Segmentationcited by 0
  4. 2026Cross-Attentive Multiview Fusion of Vision-Language Embeddingscited by 0
  5. 2026ReToken: One Token to Improve Vision-Language Models for Visual Retrievalcited by 0
  6. 2026VOIM: Training-Free Open-Vocabulary 3D Instance Mapping for RGB-D and Monocular SLAMcited by 0
  7. 2026InterHier: Learning Interconnected Hierarchical Semantics for Open-Vocabulary Object Detectioncited by 0