mlsanityData to 7 Oct 2026

Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches

arXiv 2203.14843 ↗First posted 28 Mar 2022cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

The human visual system is remarkable in learning new visual concepts from just a few examples. This is precisely the goal behind few-shot class incremental learning (FSCIL), where the emphasis is additionally placed on ensuring the model does not suffer from "forgetting". In this paper, we push the boundary further for FSCIL by addressing two key questions that bottleneck its ubiquitous application (i) can the model learn from diverse modalities other than just photo (as humans do), and (ii) what if photos are not readily accessible (due to ethical and privacy constraints). Our key innovation lies in advocating the use of sketches as a new modality for class support. The product is a "Doodle It Yourself" (DIY) FSCIL framework where the users can freely sketch a few examples of a novel class for the model to learn to recognize photos of that class. For that, we present a framework that infuses (i) gradient consensus for domain invariant learning, (ii) knowledge distillation for preserving old class information, and (iii) graph attention networks for message passing between old and novel classes. We experimentally show that sketches are better class support than text in the context of FSCIL, echoing findings elsewhere in the sketching literature.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 49 of its 71 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

49 of 71 references.

References · 49

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017Graph Attention Networkscited by 8,854
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2016Learning without Forgettingcited by 2,465
  8. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  11. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  12. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
37 more
  1. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  2. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  3. 2019A Closer Look at Few-shot Classificationcited by 679
  4. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  5. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  6. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  7. 2020Few-Shot Class-Incremental Learningcited by 249
  8. 2019Episodic Training for Domain Generalizationcited by 245
  9. 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
  10. 2021Few-Shot Incremental Learning with Continually Evolved Classifierscited by 184
  11. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 114
  12. 2017Incremental Learning Through Deep Adaptationcited by 108
  13. 2019Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulationcited by 103
  14. 2019Doodle to Search: Practical Zero-Shot Sketch-based Image Retrievalcited by 79
  15. 2020Sketchformer: Transformer-based Representation for Sketched Structurecited by 78
  16. 2018A Zero-Shot Framework for Sketch-based Image Retrievalcited by 70
  17. 2017Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrievalcited by 68
  18. 2021Domain Generalization via Gradient Surgerycited by 60
  19. 2020Deep Learning for Free-Hand Sketch: A Surveycited by 59
  20. 2020Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image Retrievalcited by 59
  21. 2019Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrievalcited by 57
  22. 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
  23. 2021StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrievalcited by 50
  24. 2018Learning to Sketch with Shortcut Cycle Consistencycited by 47
  25. 2016One-Shot Generalization in Deep Generative Modelscited by 44
  26. 2020Creative Sketch Generationcited by 44
  27. 2018Zero-Shot Sketch-Image Hashingcited by 43
  28. 2021Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwritingcited by 38
  29. 2021More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrievalcited by 38
  30. 2020Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketchescited by 37
  31. 2018SketchMate: Deep Hashing for Million-Scale Human Sketch Retrievalcited by 35
  32. 2019LiveSketch: Query Perturbations for Guided Sketch-based Visual Searchcited by 34
  33. 2020Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrievalcited by 32
  34. 2020Why Do Line Drawings Work? A Realism Hypothesiscited by 28
  35. 2020Sketch-Guided Object Localization in Natural Imagescited by 21
  36. 2020SketchEmbedNet: Learning Novel Concepts by Imitating Drawingscited by 18
  37. 2021Exploiting Aliasing for Manga Restorationcited by 10

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 10; 2024: 8; 2025: 3.

23 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2024RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketchescited by 41
  3. 2023Picture that Sketch: Photorealistic Image Generation from Abstract Sketchescited by 34
  4. 2024It's All About Your Sketch: Democratising Sketch Control in Diffusion Modelscited by 28
  5. 2023What Can Human Sketches Do for Object Detection?cited by 19
  6. 2023Sketch2Saliency: Learning to Detect Salient Objects from Human Drawingscited by 17
  7. 2022Adaptive Fine-Grained Sketch-Based Image Retrievalcited by 15
  8. 2023Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variantscited by 14
  9. 2023DemoCaricature: Democratising Caricature Generation with a Rough Sketchcited by 10
  10. 2024Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakerscited by 8
  11. 2023Data-Free Sketch-Based Image Retrievalcited by 7
  12. 2023Continual Learning through Human-Robot Interaction: Human Perceptions of a Continual Learning Robot in Repeated Interactionscited by 4
11 more
  1. 2024What Sketch Explainability Really Means for Downstream Taskscited by 4
  2. 2025SketchFusion: Learning Universal Sketch Features through Fusing Foundation Modelscited by 4
  3. 2024Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learningcited by 2
  4. 2024Do Generalised Classifiers really work on Human Drawn Sketches?cited by 2
  5. 2024Cross-Modal Few-Shot Learning: a Generative Transfer Learning Frameworkcited by 2
  6. 2022Towards Practicality of Sketch-Based Visual Understandingcited by 1
  7. 2023Sketch-based Video Object Segmentation: Benchmark and Analysiscited by 1
  8. 2025SketchYourSeg: Mask-Free Subjective Image Segmentation via Freehand Sketchescited by 1
  9. 2023Sketch-based Video Object Localizationcited by 0
  10. 2024SketchDeco: Training-Free Latent Composition for Precise Sketch Colourisationcited by 0
  11. 2025Doodle Your Keypoints: Sketch-Based Few-Shot Keypoint Detectioncited by 0