mlsanityData to 13 Oct 2026

Partially Does It: Towards Scene-Level FG-SBIR with Partial Input

arXiv 2203.14804 ↗First posted 28 Mar 2022cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

We scrutinise an important observation plaguing scene-level sketch research -- that a significant portion of scene sketches are "partial". A quick pilot study reveals: (i) a scene sketch does not necessarily contain all objects in the corresponding photo, due to the subjective holistic interpretation of scenes, (ii) there exists significant empty (white) regions as a result of object-level abstraction, and as a result, (iii) existing scene-level fine-grained sketch-based image retrieval methods collapse as scene sketches become more partial. To solve this "partial" problem, we advocate for a simple set-based approach using optimal transport (OT) to model cross-modal region associativity in a partially-aware fashion. Importantly, we improve upon OT to further account for holistic partialness by comparing intra-modal adjacency matrices. Our proposed method is not only robust to partial scene-sketches but also yields state-of-the-art performance on existing datasets.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Builds on 49 of its 91 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

49 of 91 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,106
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,457
  3. 2021Convex optimizationcited by 2,488
  4. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,316
  5. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 861
  6. 2013Network In Networkcited by 603
  7. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 592
  8. 2016LIFT: Learned Invariant Feature Transformcited by 305
  9. 2018DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptationcited by 213
  10. 2018Optimal Transport for structured data with application on graphscited by 171
  11. 2019Pluralistic Image Completioncited by 162
  12. 2018LF-Net: Learning Local Features from Imagescited by 151
37 more
  1. 2019Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filterscited by 117
  2. 2019CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrievalcited by 110
  3. 2020Graph Optimal Transport for Cross-Domain Alignmentcited by 103
  4. 2021Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformerscited by 93
  5. 2019Joint-task Self-supervised Learning for Temporal Correspondencecited by 80
  6. 2020Self-EMD: Self-Supervised Object Detection without ImageNetcited by 70
  7. 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 68
  8. 2020SketchyCOCO: Image Generation from Freehand Scene Sketchescited by 67
  9. 2021The Spatially-Correlative Loss for Various Image Translation Taskscited by 66
  10. 2019GOT: An Optimal Transport framework for Graph comparisoncited by 64
  11. 2015Learning Descriptors for Object Recognition and 3D Pose Estimationcited by 60
  12. 2020Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrievalcited by 59
  13. 2020Vocabulary Learning via Optimal Transport for Neural Machine Translationcited by 58
  14. 2014TILDE: A Temporally Invariant Learned DEtectorcited by 55
  15. 2019Interactive Sketch & Fill: Multiclass Sketch-to-Image Translationcited by 51
  16. 2021TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformationscited by 48
  17. 2018Learning to Sketch with Shortcut Cycle Consistencycited by 47
  18. 2017Deep Network Flow for Multi-Object Trackingcited by 45
  19. 2020Creative Sketch Generationcited by 44
  20. 2020Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestaltcited by 42
  21. 2021Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwritingcited by 38
  22. 2021More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrievalcited by 38
  23. 2016Quad-networks: unsupervised learning to rank for interest point detectioncited by 36
  24. 2018Learning Deep Sketch Abstractioncited by 36
  25. 2019LiveSketch: Query Perturbations for Guided Sketch-based Visual Searchcited by 34
  26. 2018SketchyScene: Richly-Annotated Scene Sketchescited by 32
  27. 2020Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrievalcited by 32
  28. 2021Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrievalcited by 28
  29. 2016Learning Covariant Feature Detectorscited by 25
  30. 2018On the Differentiability of the Solution to Convex Optimization Problemscited by 23
  31. 2021Image Inpainting with External-internal Learning and Monochromic Bottleneckcited by 21
  32. 2021SketchLattice: Latticed Representation for Sketch Manipulationcited by 19
  33. 2020COPT: Coordinated Optimal Transport for Graph Sketchingcited by 17
  34. 2020Go Wide, Then Narrow: Efficient Training of Deep Thin Networkscited by 12
  35. 2018Sketch-R2CNN: An Attentive Network for Vector Sketch Recognitioncited by 6
  36. 2018End-to-end learning of keypoint detector and descriptor for pose invariant 3D matchingcited by 4
  37. 2021Pix2Point: Learning Outdoor 3D Using Sparse Point Clouds and Optimal Transportcited by 4

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 5; 2023: 5; 2024: 6; 2025: 3; 2026: 1.

20 papers.

Sort:

By area:

  1. 2023Picture that Sketch: Photorealistic Image Generation from Abstract Sketchescited by 34
  2. 2022SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Textcited by 30
  3. 2024It's All About Your Sketch: Democratising Sketch Control in Diffusion Modelscited by 29
  4. 2023What Can Human Sketches Do for Object Detection?cited by 19
  5. 2023Sketch2Saliency: Learning to Detect Salient Objects from Human Drawingscited by 17
  6. 2022Adaptive Fine-Grained Sketch-Based Image Retrievalcited by 15
  7. 2024You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrievalcited by 12
  8. 2024How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?cited by 10
  9. 2024Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakerscited by 8
  10. 2022Cross-Modal Fusion Distillation for Fine-Grained Sketch-Based Image Retrievalcited by 4
  11. 2024What Sketch Explainability Really Means for Downstream Taskscited by 4
  12. 2025SketchFusion: Learning Universal Sketch Features through Fusing Foundation Modelscited by 4
8 more
  1. 2024Freeview Sketching: View-Aware Fine-Grained Sketch-Based Image Retrievalcited by 2
  2. 2025Sketch Down the FLOPs: Towards Efficient Networks for Human Sketchcited by 2
  3. 2022Towards Practicality of Sketch-Based Visual Understandingcited by 1
  4. 2023Sketch-based Video Object Segmentation: Benchmark and Analysiscited by 1
  5. 2026ELViS: Efficient Visual Similarity from Local Descriptors that Generalizes Across Domainscited by 1
  6. 2022Multimodal Query-guided Object Localizationcited by 0
  7. 2023Symmetrical Bidirectional Knowledge Alignment for Zero-Shot Sketch-Based Image Retrievalcited by 0
  8. 2025Doodle Your Keypoints: Sketch-Based Few-Shot Keypoint Detectioncited by 0