mlsanityData to 7 Oct 2026

Sketch-based Video Object Segmentation: Benchmark and Analysis

arXiv 2311.07261 ↗First posted 13 Nov 2023cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Reference-based video object segmentation is an emerging topic which aims to segment the corresponding target object in each video frame referred by a given reference, such as a language expression or a photo mask. However, language expressions can sometimes be vague in conveying an intended concept and ambiguous when similar objects in one frame are hard to distinguish by language. Meanwhile, photo masks are costly to annotate and less practical to provide in a real application. This paper introduces a new task of sketch-based video object segmentation, an associated benchmark, and a strong baseline. Our benchmark includes three datasets, Sketch-DAVIS16, Sketch-DAVIS17 and Sketch-YouTube-VOS, which exploit human-drawn sketches as an informative yet low-cost reference for video object segmentation. We take advantage of STCN, a popular baseline of semi-supervised VOS task, and evaluate what the most effective design for incorporating a sketch reference is. Experimental results show sketch is more effective yet annotation-efficient than other references, such as photo masks, language and scribble.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 47 of its 58 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

47 of 58 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  4. 2016HyperNetworkscited by 1,003
  5. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  6. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  7. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  8. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 330
  9. 2017A Neural Representation of Sketch Drawingscited by 311
  10. 2016One-Shot Video Object Segmentationcited by 233
  11. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 228
  12. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
35 more
  1. 2022Language as Queries for Referring Video Object Segmentationcited by 180
  2. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  3. 2018Video Object Segmentation with Language Referring Expressionscited by 161
  4. 2021End-to-End Referring Video Object Segmentation with Multimodal Transformerscited by 135
  5. 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
  6. 2017SegFlow: Joint Learning for Video Object Segmentation and Optical Flowcited by 92
  7. 2020Zero-Shot Video Object Segmentation via Attentive Graph Neural Networkscited by 84
  8. 2020Collaborative Video Object Segmentation by Multi-Scale Foreground-Background Integrationcited by 81
  9. 2019Doodle to Search: Practical Zero-Shot Sketch-based Image Retrievalcited by 79
  10. 2020Sketchformer: Transformer-based Representation for Sketched Structurecited by 78
  11. 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
  12. 2020Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrievalcited by 59
  13. 2018Fast and Accurate Online Video Object Segmentation via Tracking Partscited by 57
  14. 2019Anchor Diffusion for Unsupervised Video Object Segmentationcited by 50
  15. 2021StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrievalcited by 50
  16. 2022Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentationcited by 50
  17. 2018The 2018 DAVIS Challenge on Video Object Segmentationcited by 47
  18. 2022FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Contextcited by 47
  19. 2020Memory Aggregation Networks for Efficient Interactive Video Object Segmentationcited by 40
  20. 2021More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrievalcited by 38
  21. 2022Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentationcited by 37
  22. 2022Learning Quality-aware Dynamic Memory for Video Object Segmentationcited by 35
  23. 2019LiveSketch: Query Perturbations for Guided Sketch-based Visual Searchcited by 34
  24. 2021SketchEdit: Mask-Free Local Image Manipulation with Partial Sketchescited by 32
  25. 2022Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrievalcited by 27
  26. 2022SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted Expectation-Maximizationcited by 26
  27. 2022Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketchescited by 23
  28. 2022Global Spectral Filter Memory Network for Video Object Segmentationcited by 22
  29. 2020Sketch-Guided Object Localization in Natural Imagescited by 21
  30. 2021Guided Interactive Video Object Segmentation Using Reliability-Based Attention Mapscited by 21
  31. 2022Partially Does It: Towards Scene-Level FG-SBIR with Partial Inputcited by 20
  32. 2021Learning to Recommend Frame for Interactive Video Object Segmentation in the Wildcited by 18
  33. 2022Tackling Background Distraction in Video Object Segmentationcited by 16
  34. 2020ALBA : Reinforcement Learning for Video Object Segmentationcited by 9
  35. 2018Sketch-a-Classifier: Sketch-based Photo Classifier Generationcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025SketchAnimator: Animate Sketch via Motion Customization of Text-to-Video Diffusion Modelscited by 0