mlsanityData to 14 Oct 2026

Localizing Infinity-shaped fishes: Sketch-guided object localization in the wild

arXiv 2109.11874 ↗First posted 24 Sep 2021cs.CV cs.GRcited by 3

Not on the map. Why?

Abstract (arXiv)

This work investigates the problem of sketch-guided object localization (SGOL), where human sketches are used as queries to conduct the object localization in natural images. In this cross-modal setting, we first contribute with a tough-to-beat baseline that without any specific SGOL training is able to outperform the previous works on a fixed set of classes. The baseline is useful to analyze the performance of SGOL approaches based on available simple yet powerful methods. We advance prior arts by proposing a sketch-conditioned DETR (DEtection TRansformer) architecture which avoids a hard classification and alleviates the domain gap between sketches and images to localize object instances. Although the main goal of SGOL is focused on object detection, we explored its natural extension to sketch-guided instance segmentation. This novel task allows to move towards identifying the objects at pixel level, which is of key importance in several applications. We experimentally demonstrate that our model and its variants significantly advance over previous state-of-the-art results. All training and testing code of our model will be released to facilitate future research{{https://github.com/priba/sgol_wild}}.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 41 of its 59 references that are in the map. First posted on arXiv 24 Sep 2021.

Builds on

41 of 59 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2017Mask R-CNNcited by 7,277
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  10. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  11. 2015Fast R-CNNcited by 2,933
  12. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
29 more
  1. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  2. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  3. 2018Panoptic Segmentationcited by 802
  4. 2018Path Aggregation Network for Instance Segmentationcited by 798
  5. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  6. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 540
  7. 2020ResNeSt: Split-Attention Networkscited by 445
  8. 2016R-FCN: Object Detection via Region-based Fully Convolutional Networkscited by 414
  9. 2017One-Shot Learning for Semantic Segmentationcited by 337
  10. 2019Mask Scoring R-CNNcited by 266
  11. 2016One-Shot Video Object Segmentationcited by 235
  12. 2020DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolutioncited by 222
  13. 2014Simultaneous Detection and Segmentationcited by 159
  14. 2019Bottom-up Object Detection by Grouping Extreme and Center Pointscited by 156
  15. 2019Data augmentation using learned transformations for one-shot medical image segmentationcited by 138
  16. 2015Instance-aware Semantic Segmentation via Multi-task Network Cascadescited by 121
  17. 2019Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidthcited by 89
  18. 2019SSAP: Single-Shot Instance Segmentation With Affinity Pyramidcited by 85
  19. 2019Doodle to Search: Practical Zero-Shot Sketch-based Image Retrievalcited by 79
  20. 2017Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrievalcited by 68
  21. 2019One-Shot Object Detection with Co-Attention and Co-Excitationcited by 67
  22. 2020Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrievalcited by 59
  23. 2019SpineNet: Learning Scale-Permuted Backbone for Recognition and Localizationcited by 50
  24. 2018Zero-Shot Sketch-Image Hashingcited by 43
  25. 2018One-Shot Instance Segmentationcited by 43
  26. 2015Sketch-based 3D Shape Retrieval using Convolutional Neural Networkscited by 41
  27. 2020Sketch-Guided Object Localization in Natural Imagescited by 21
  28. 2018One-Shot Segmentation in Cluttercited by 7
  29. 2018Learning Cross-Modal Deep Embeddings for Multi-Object Image Retrieval using Text and Sketchcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 3.

3 papers.

  1. 2023What Can Human Sketches Do for Object Detection?cited by 19
  2. 2023Query-guided Attention in Vision Transformers for Localizing Objects Using a Single Sketchcited by 3
  3. 2023Sketch-based Video Object Localizationcited by 0