# Deep Self-Supervised Representation Learning for Free-Hand Sketch

arXiv: https://arxiv.org/abs/2002.00867 · HTML: https://mlsanity.com/paper/2002.00867/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2020.

## Built on by (most cited first, 7 of 7)

- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [Boosting Point-BERT by Multi-choice Tokens](https://mlsanity.com/paper/2207.13226/) (2022)
- [Extending Momentum Contrast with Cross Similarity Consistency Regularization](https://mlsanity.com/paper/2206.04676/) (2022)
- [Equipping Sketch Patches with Context-Aware Positional Encoding for Graphic Sketch Representation](https://mlsanity.com/paper/2403.17525/) (2024)
- [Leveraging Unlabeled Data for Sketch-based Understanding](https://mlsanity.com/paper/2204.12522/) (2022)
- [BoxSeg: Quality-Aware and Peer-Assisted Learning for Box-supervised Instance Segmentation](https://mlsanity.com/paper/2504.05137/) (2025)
- [Zero-shot sketch-based remote sensing image retrieval based on multi-level and attention-guided tokenization](https://mlsanity.com/paper/2402.02141/) (2024)

## Abstract (arXiv)

In this paper, we tackle for the first time, the problem of self-supervised representation learning for free-hand sketches. This importantly addresses a common problem faced by the sketch community -- that annotated supervisory data are difficult to obtain. This problem is very challenging in that sketches are highly abstract and subject to different drawing styles, making existing solutions tailored for photos unsuitable. Key for the success of our self-supervised learning paradigm lies with our sketch-specific designs: (i) we propose a set of pretext tasks specifically designed for sketches that mimic different drawing styles, and (ii) we further exploit the use of a textual convolution network (TCN) in a dual-branch architecture for sketch feature learning, as means to accommodate the sequential stroke nature of sketches. We demonstrate the superiority of our sketch-specific designs through two sketch-related applications (retrieval and recognition) on a million-scale sketch dataset, and show that the proposed approach outperforms the state-of-the-art unsupervised representation learning methods, and significantly narrows the performance gap between with supervised representation learning.