mlsanityData to 14 Oct 2026

YORO -- Lightweight End to End Visual Grounding

arXiv 2211.07912 ↗First posted 15 Nov 2022cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

We present YORO - a multi-modal transformer encoder-only architecture for the Visual Grounding (VG) task. This task involves localizing, in an image, an object referred via natural language. Unlike the recent trend in the literature of using multi-stage approaches that sacrifice speed for accuracy, YORO seeks a better trade-off between speed an accuracy by embracing a single-stage design, without CNN backbone. YORO consumes natural language queries, image patches, and learnable detection tokens and predicts coordinates of the referred object, using a single transformer encoder. To assist the alignment between text and visual objects, a novel patch-text alignment loss is proposed. Extensive experiments are conducted on 5 different datasets with ablations on architecture design choices. YORO is shown to support real-time inference and outperform all approaches in this class (single-stage methods) by large margins. It is also the fastest VG model and achieves the best speed/accuracy trade-off in the literature.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 78 of its 91 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

78 of 91 references.

References · 78

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  9. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  10. 2020End-to-End Object Detection with Transformerscited by 7,414
  11. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  12. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
38 more
  1. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  5. 2018YOLOv3: An Incremental Improvementcited by 2,659
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  8. 2019Searching for MobileNetV3cited by 1,994
  9. 2021Vision Transformers for Dense Predictioncited by 1,803
  10. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  11. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,630
  12. 2016YOLO9000: Better, Faster, Strongercited by 1,431
  13. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  14. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  15. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  16. 2016Modeling Context in Referring Expressionscited by 1,063
  17. 2021Transformers in Vision: A Surveycited by 1,013
  18. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  19. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  20. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  21. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 459
  22. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  23. 2018Stacked Cross Attention for Image-Text Matchingcited by 416
  24. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 412
  25. 2018Deep Learning for Generic Object Detection: A Surveycited by 392
  26. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  27. 201912-in-1: Multi-Task Vision and Language Representation Learningcited by 278
  28. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
  29. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  30. 2016Modeling Context Between Objects for Referring Expression Understandingcited by 251
  31. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  32. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  33. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 222
  34. 2020ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graphcited by 211
  35. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  36. 2016Speed/accuracy trade-offs for modern convolutional object detectorscited by 198
  37. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 195
  38. 2019Visual Semantic Reasoning for Image-Text Matchingcited by 185

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 5; 2025: 3; 2026: 2.

17 papers.

Sort:

By area:

  1. 2024Toward Visual Grounding: A Surveycited by 52
  2. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  3. 2023CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Groundingcited by 35
  4. 2023Language Adaptive Weight Generation for Multi-task Visual Groundingcited by 35
  5. 2024HiVG: Hierarchical Multimodal Fine-grained Modulation for Visual Groundingcited by 31
  6. 2024OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modelingcited by 28
  7. 2024Enhancing Vision-Language Pre-training with Rich Supervisionscited by 13
  8. 2025WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentationcited by 9
  9. 2025SwimVG: Step-wise Multimodal Fusion and Adaption for Visual Groundingcited by 8
  10. 2023Single-Stage Visual Relationship Learning using Conditional Queriescited by 7
  11. 2024MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehensioncited by 6
  12. 2023Benchmarking Diverse-Modal Entity Linking with Generative Modelscited by 5
5 more
  1. 2023Toward Unsupervised Realistic Visual Question Answeringcited by 4
  2. 2023Multiple-Question Multiple-Answer Text-VQAcited by 4
  3. 2025LIHE: Linguistic Instance-Split Hyperbolic-Euclidean Framework for Generalized Weakly-Supervised Referring Expression Comprehensioncited by 1
  4. 2026BARE: Towards Bias-Aware and Reasoning-Enhanced One-Tower Visual Groundingcited by 1
  5. 2026STP-GUI: Spatially-Aware Token Pruning for Efficient Web-GUI Agentscited by 0