mlsanityData to 7 Oct 2026

SeqTR: A Simple yet Universal Network for Visual Grounding

arXiv 2203.16265 ↗First posted 30 Mar 2022cs.CVcited by 138

Not on the map. Why?

Cited by 21 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we propose a simple yet universal network termed SeqTR for visual grounding tasks, e.g., phrase localization, referring expression comprehension (REC) and segmentation (RES). The canonical paradigms for visual grounding often require substantial expertise in designing network architectures and loss functions, making them hard to generalize across tasks. To simplify and unify the modeling, we cast visual grounding as a point prediction problem conditioned on image and text inputs, where either the bounding box or binary mask is represented as a sequence of discrete coordinate tokens. Under this paradigm, visual grounding tasks are unified in our SeqTR network without task-specific branches or heads, e.g., the convolutional mask decoder for RES, which greatly reduces the complexity of multi-task modeling. In addition, SeqTR also shares the same optimization objective for all tasks with a simple cross-entropy loss, further reducing the complexity of deploying hand-crafted loss functions. Experiments on five benchmark datasets demonstrate that the proposed SeqTR outperforms (or is on par with) the existing state-of-the-arts, proving that a simple yet universal approach for visual grounding is indeed feasible. Source code is available at https://github.com/sean-zhuh/SeqTR.

Key facts. Cited by 138 papers in the map (data through 7 Oct 2026). Builds on 54 of its 66 references that are in the map. First posted on arXiv 30 Mar 2022.

Builds on

54 of 66 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  9. 2016Wide Residual Networkscited by 3,057
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  11. 2018YOLOv3: An Incremental Improvementcited by 2,652
  12. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
38 more
  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  2. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  3. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  4. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  5. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  6. 2016Modeling Context in Referring Expressionscited by 1,054
  7. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  8. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  9. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  10. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  11. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  12. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  13. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  14. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  15. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  16. 2016Modeling Context Between Objects for Referring Expression Understandingcited by 250
  17. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  18. 2020ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graphcited by 211
  19. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  20. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 194
  21. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  22. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 139
  23. 2020Improving One-stage Visual Grounding by Recursive Sub-query Constructioncited by 137
  24. 2020Referring Image Segmentation via Cross-Modal Progressive Comprehensioncited by 131
  25. 2017Learning Two-Branch Neural Networks for Image-Text Matching Taskscited by 119
  26. 2019A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehensioncited by 111
  27. 2019Dynamic Graph Attention for Referring Expression Comprehensioncited by 111
  28. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  29. 2016Modeling Relationships in Referential Expressions with Compositional Modular Networkscited by 106
  30. 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
  31. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  32. 2019Learning to Compose and Reason with Language Tree Structures for Visual Groundingcited by 98
  33. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  34. 2019Improving Referring Expression Grounding with Cross-modal Attention-guided Erasingcited by 88
  35. 2017Grounding Referring Expressions in Images by Variational Contextcited by 82
  36. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  37. 2017Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queriescited by 56
  38. 2021Look Before You Leap: Learning Landmark Features for One-Stage Visual Groundingcited by 51

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 43; 2024: 41; 2025: 37; 2026: 11.

138 papers.

Sort:

By area:

  1. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  2. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
  3. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  4. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
  5. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  6. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  7. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  8. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  9. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  10. 2023LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Modelscited by 113
  11. 2023Contextual Object Detection with Multimodal Large Language Modelscited by 100
  12. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Groundingcited by 78
38 more
  1. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  2. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  3. 2024SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentationcited by 68
  4. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  5. 2023PerceptionGPT: Effectively Fusing Visual Perception into LLMcited by 59
  6. 2023Aligning and Prompting Everything All at Once for Universal Visual Perceptioncited by 59
  7. 2023Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Modelcited by 52
  8. 2024Toward Visual Grounding: A Surveycited by 52
  9. 2023Towards General Visual-Linguistic Face Forgery Detectioncited by 45
  10. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  11. 2023Mask Grounding for Referring Image Segmentationcited by 40
  12. 2024ReMamber: Referring Image Segmentation with Mamba Twistercited by 38
  13. 2023A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysiscited by 37
  14. 2024SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal Fusioncited by 36
  15. 2023Language Adaptive Weight Generation for Multi-task Visual Groundingcited by 35
  16. 2023Referring Image Segmentation Using Text Supervisioncited by 35
  17. 2023Medical Phrase Grounding with Region-Phrase Context Contrastive Alignmentcited by 32
  18. 2024HiVG: Hierarchical Multimodal Fine-grained Modulation for Visual Groundingcited by 31
  19. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 31
  20. 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
  21. 2024OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modelingcited by 28
  22. 2025MMR: A Large-scale Benchmark Dataset for Multi-target and Multi-granularity Reasoning Segmentationcited by 28
  23. 2024Unifying Visual and Vision-Language Tracking via Contrastive Learningcited by 27
  24. 2022DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Groundingcited by 24
  25. 2023RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentationcited by 24
  26. 2023u-LLaVA: Unifying Multi-Modal Tasks via Large Language Modelcited by 24
  27. 2024Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Modelscited by 24
  28. 2023Context Disentangling and Prototype Inheriting for Robust Visual Groundingcited by 23
  29. 2023UniRef++: Segment Every Reference Object in Spatial and Temporal Spacescited by 22
  30. 2025Multimodal Referring Segmentation: A Surveycited by 22
  31. 2023Unveiling Parts Beyond Objects:Towards Finer-Granularity Referring Expression Segmentationcited by 21
  32. 2024An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual Groundingcited by 21
  33. 2024Cross-aware Early Fusion with Stage-divided Vision and Language Transformer Encoders for Referring Image Segmentationcited by 21
  34. 2025HiMTok: Learning Hierarchical Mask Tokens for Image Segmentation with Large Multimodal Modelcited by 19
  35. 2023Meta Compositional Referring Expression Segmentationcited by 18
  36. 2024ScanFormer: Referring Expression Comprehension by Iteratively Scanningcited by 18
  37. 2024Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression Comprehensioncited by 17
  38. 2025Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraintscited by 17

50 most cited of 138 All 138 (JSON)