mlsanityData to 7 Oct 2026

Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings

arXiv 2303.11502 ↗First posted 20 Mar 2023cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Human sketch has already proved its worth in various visual understanding tasks (e.g., retrieval, segmentation, image-captioning, etc). In this paper, we reveal a new trait of sketches - that they are also salient. This is intuitive as sketching is a natural attentive process at its core. More specifically, we aim to study how sketches can be used as a weak label to detect salient objects present in an image. To this end, we propose a novel method that emphasises on how "salient object" could be explained by hand-drawn sketches. To accomplish this, we introduce a photo-to-sketch generation model that aims to generate sequential sketch coordinates corresponding to a given visual photo through a 2D attention mechanism. Attention maps accumulated across the time steps give rise to salient regions in the process. Extensive quantitative and qualitative experiments prove our hypothesis and delineate how our sketch-based saliency detection model gives a competitive performance compared to the state-of-the-art.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 58 of its 91 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

58 of 91 references.

References · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2017Squeeze-and-Excitation Networkscited by 4,763
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  6. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  7. 2018Self-Attention Generative Adversarial Networkscited by 809
  8. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
  9. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  10. 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
  11. 2017A Neural Representation of Sketch Drawingscited by 311
  12. 2020Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentationcited by 299
38 more
  1. 2017Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approachcited by 201
  2. 2014The Secrets of Salient Object Segmentationcited by 200
  3. 2015Visual Saliency Based on Multiscale Deep Featurescited by 185
  4. 2019Salient Object Detection in the Deep Learning Era: An In-Depth Surveycited by 163
  5. 2018Reverse Attention for Salient Object Detectioncited by 113
  6. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  7. 2020Weakly-Supervised Salient Object Detection via Scribble Annotationscited by 96
  8. 2014Salient Object Detection: A Discriminative Regional Feature Integration Approachcited by 89
  9. 2020Sketchformer: Transformer-based Representation for Sketched Structurecited by 78
  10. 2018A Zero-Shot Framework for Sketch-based Image Retrievalcited by 70
  11. 2020SketchyCOCO: Image Generation from Freehand Scene Sketchescited by 67
  12. 2020Deep Learning for Free-Hand Sketch: A Surveycited by 59
  13. 2020Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrievalcited by 59
  14. 2019Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrievalcited by 57
  15. 2019Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentationcited by 57
  16. 2020Structure-Consistent Weakly Supervised Salient Object Detection with Local Saliency Coherencecited by 55
  17. 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
  18. 2019Interactive Sketch & Fill: Multiclass Sketch-to-Image Translationcited by 51
  19. 2021StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrievalcited by 50
  20. 2018Salient Objects in Clutter: Bringing Salient Object Detection to the Foregroundcited by 47
  21. 2018Learning to Sketch with Shortcut Cycle Consistencycited by 47
  22. 2022FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Contextcited by 47
  23. 2020Creative Sketch Generationcited by 44
  24. 2018Zero-Shot Sketch-Image Hashingcited by 43
  25. 2019Multi-source weak supervision for saliency detectioncited by 43
  26. 2019Unsupervised Sketch-to-Photo Synthesiscited by 38
  27. 2021Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwritingcited by 38
  28. 2021More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrievalcited by 38
  29. 2020Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketchescited by 37
  30. 2019LiveSketch: Query Perturbations for Guided Sketch-based Visual Searchcited by 34
  31. 2022Sketch3T: Test-Time Training for Zero-Shot SBIRcited by 34
  32. 2017Exploiting saliency for object segmentation from image level labelscited by 33
  33. 2018SketchyScene: Richly-Annotated Scene Sketchescited by 32
  34. 2020Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrievalcited by 32
  35. 2022SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Textcited by 30
  36. 2020Why Do Line Drawings Work? A Realism Hypothesiscited by 28
  37. 2015DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detectioncited by 24
  38. 2021Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognitioncited by 24

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 7; 2026: 2.

17 papers.

Sort:

By area:

  1. 2023CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Notcited by 79
  2. 2024RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketchescited by 41
  3. 2024It's All About Your Sketch: Democratising Sketch Control in Diffusion Modelscited by 28
  4. 2023What Can Human Sketches Do for Object Detection?cited by 19
  5. 2023Exploiting Unlabelled Photos for Stronger Fine-Grained SBIRcited by 14
  6. 2024How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?cited by 10
  7. 2025Sketch-to-Skill: Bootstrapping Robot Learning with Human Drawn Trajectory Sketchescited by 9
  8. 2025SketchFusion: Learning Universal Sketch Features through Fusing Foundation Modelscited by 4
  9. 2025SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketchescited by 3
  10. 2024Multi-modal Generation via Cross-Modal In-Context Learningcited by 2
  11. 2025Sketch Down the FLOPs: Towards Efficient Networks for Human Sketchcited by 2
  12. 2025SketchYourSeg: Mask-Free Subjective Image Segmentation via Freehand Sketchescited by 1
5 more
  1. 2026Stroke of Surprise: Progressive Semantic Illusions in Vector Sketchingcited by 1
  2. 2024SketchDeco: Training-Free Latent Composition for Precise Sketch Colourisationcited by 0
  3. 2025Breaking the Box: Enhancing Remote Sensing Image Segmentation with Freehand Sketchescited by 0
  4. 2025Doodle Your Keypoints: Sketch-Based Few-Shot Keypoint Detectioncited by 0
  5. 2026Focus Where It Counts: A Salience-Driven Vision-Language Model for Low Vision Assistancecited by 0