# Enhance Sketch Recognition's Explainability via Semantic Component-Level Parsing

arXiv: https://arxiv.org/abs/2312.07875 · HTML: https://mlsanity.com/paper/2312.07875/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [A Neural Representation of Sketch Drawings](https://mlsanity.com/paper/1704.03477/) (2017)
- [Interpretable Counterfactual Explanations Guided by Prototypes](https://mlsanity.com/paper/1907.02584/) (2019)
- [Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models](https://mlsanity.com/paper/1908.01224/) (2019)
- [Deep Learning for Free-Hand Sketch: A Survey](https://mlsanity.com/paper/2001.02600/) (2020)
- [Memory-Based Graph Networks](https://mlsanity.com/paper/2002.09518/) (2020)
- [Interpreting CNN Knowledge via an Explanatory Graph](https://mlsanity.com/paper/1708.01785/) (2017)
- [SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval](https://mlsanity.com/paper/1804.01401/) (2018)
- [Multi-Graph Transformer for Free-Hand Sketch Recognition](https://mlsanity.com/paper/1912.11258/) (2019)
- [A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts](https://mlsanity.com/paper/2105.00290/) (2021)
- [One Explanation is Not Enough: Structured Attention Graphs for Image Classification](https://mlsanity.com/paper/2011.06733/) (2020)
- [Universal Perceptual Grouping](https://mlsanity.com/paper/1808.02312/) (2018)
- [Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networks](https://mlsanity.com/paper/2203.03282/) (2022)
- [Enabling My Robot To Play Pictionary : Recurrent Neural Networks For Sketch Recognition](https://mlsanity.com/paper/1608.03369/) (2016)
- [Sketch Input Method Editor: A Comprehensive Dataset and Methodology for Systematic Input Recognition](https://mlsanity.com/paper/2311.18254/) (2023)

## Built on by (most cited first, 1 of 1)

- [GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understanding](https://mlsanity.com/paper/2409.03258/) (2024)

## Abstract (arXiv)

Free-hand sketches are appealing for humans as a universal tool to depict the visual world. Humans can recognize varied sketches of a category easily by identifying the concurrence and layout of the intrinsic semantic components of the category, since humans draw free-hand sketches based a common consensus that which types of semantic components constitute each sketch category. For example, an airplane should at least have a fuselage and wings. Based on this analysis, a semantic component-level memory module is constructed and embedded in the proposed structured sketch recognition network in this paper. The memory keys representing semantic components of each sketch category can be self-learned and enhance the recognition network's explainability. Our proposed networks can deal with different situations of sketch recognition, i.e., with or without semantic components labels of strokes. Experiments on the SPG and SketchIME datasets demonstrate the memory module's flexibility and the recognition network's explainability. The code and data are available at https://github.com/GuangmingZhu/SketchESC.