mlsanityData to 7 Oct 2026

What Sketch Explainability Really Means for Downstream Tasks

arXiv 2403.09480 ↗First posted 14 Mar 2024cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we explore the unique modality of sketch for explainability, emphasising the profound impact of human strokes compared to conventional pixel-oriented studies. Beyond explanations of network behavior, we discern the genuine implications of explainability across diverse downstream sketch-related tasks. We propose a lightweight and portable explainability solution -- a seamless plugin that integrates effortlessly with any pre-trained model, eliminating the need for re-training. Demonstrating its adaptability, we present four applications: highly studied retrieval and generation, and completely novel assisted drawing and sketch adversarial attacks. The centrepiece to our solution is a stroke-level attribution map that takes different forms when linked with downstream tasks. By addressing the inherent non-differentiability of rasterisation, we enable explanations at both coarse stroke level (SLA) and partial stroke level (P-SLA), each with its advantages for specific downstream tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 89 of its 120 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

89 of 120 references.

References · 89

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  10. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  11. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  12. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
38 more
  1. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  2. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  3. 2018Sanity Checks for Saliency Mapscited by 1,149
  4. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  5. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  6. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  7. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  8. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  9. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  10. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  11. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  12. 2017The (Un)reliability of saliency methodscited by 331
  13. 2016Top-down Neural Attention by Excitation Backpropcited by 318
  14. 2017A Neural Representation of Sketch Drawingscited by 311
  15. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  16. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  17. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
  18. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  19. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
  20. 2020Debugging Tests for Model Explanationscited by 146
  21. 2020Physically Realizable Adversarial Examples for LiDAR Object Detectioncited by 137
  22. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  23. 2017On the Robustness of Semantic Segmentation Models to Adversarial Attackscited by 126
  24. 2019XRAI: Better Attributions Through Regionscited by 116
  25. 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
  26. 2019Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesiscited by 106
  27. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  28. 2021Guided Integrated Gradients: An Adaptive Path Method for Removing Noisecited by 102
  29. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  30. 2022B-cos Networks: Alignment is All We Need for Interpretabilitycited by 88
  31. 2018SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesiscited by 81
  32. 2019Doodle to Search: Practical Zero-Shot Sketch-based Image Retrievalcited by 79
  33. 2023CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Notcited by 79
  34. 2018How Important Is a Neuron?cited by 73
  35. 2019An Evaluation of the Human-Interpretability of Explanationcited by 71
  36. 2018A Zero-Shot Framework for Sketch-based Image Retrievalcited by 70
  37. 2017Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrievalcited by 68
  38. 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 67

50 most cited of 89 All 89 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 3.

4 papers.

  1. 2024LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Modelcited by 5
  2. 2026Multi-View Hierarchical Graph Neural Network for Sketch-Based 3D Shape Retrievalcited by 0
  3. 2026Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrievalcited by 0
  4. 2026SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketchescited by 0