# CLIPasso: Semantically-Aware Object Sketching

arXiv: https://arxiv.org/abs/2202.05822 · HTML: https://mlsanity.com/paper/2202.05822/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 58 references that are in the map.
- First posted on arXiv 11 Feb 2022.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [A Neural Representation of Sketch Drawings](https://mlsanity.com/paper/1704.03477/) (2017)
- [CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders](https://mlsanity.com/paper/2106.14843/) (2021)
- [Sketchformer: Transformer-based Representation for Sketched Structure](https://mlsanity.com/paper/2002.10381/) (2020)
- [Synthesizing Programs for Images using Reinforced Adversarial Learning](https://mlsanity.com/paper/1804.01118/) (2018)
- [Photo-Sketching: Inferring Contour Drawings from Images](https://mlsanity.com/paper/1901.00542/) (2019)
- [SketchyCOCO: Image Generation from Freehand Scene Sketches](https://mlsanity.com/paper/2003.02683/) (2020)
- [Deep Learning for Free-Hand Sketch: A Survey](https://mlsanity.com/paper/2001.02600/) (2020)
- [Learning to Sketch with Shortcut Cycle Consistency](https://mlsanity.com/paper/1805.00247/) (2018)
- [Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt](https://mlsanity.com/paper/2005.09159/) (2020)
- [Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts](https://mlsanity.com/paper/2109.08857/) (2021)
- [Learning Deep Sketch Abstraction](https://mlsanity.com/paper/1804.04804/) (2018)
- [Why Do Line Drawings Work? A Realism Hypothesis](https://mlsanity.com/paper/2002.06260/) (2020)
- [Learning to Draw: Emergent Communication through Sketching](https://mlsanity.com/paper/2106.02067/) (2021)
- [Unsupervised Doodling and Painting with Improved SPIRAL](https://mlsanity.com/paper/1910.01007/) (2019)
- [Differentiable Drawing and Sketching](https://mlsanity.com/paper/2103.16194/) (2021)
- [Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories](https://mlsanity.com/paper/1709.04121/) (2017)
- [SketchLattice: Latticed Representation for Sketch Manipulation](https://mlsanity.com/paper/2108.11636/) (2021)
- [Free-hand Sketch Synthesis with Deformable Stroke Models](https://mlsanity.com/paper/1510.02644/) (2015)
- [Emergent Graphical Conventions in a Visual Communication Game](https://mlsanity.com/paper/2111.14210/) (2021)
- [Pragmatic inference and visual abstraction enable contextual flexibility during visual communication](https://mlsanity.com/paper/1903.04448/) (2019)
- [Synthesizing human-like sketches from natural images using a conditional convolutional decoder](https://mlsanity.com/paper/2003.07101/) (2020)

## Built on by (most cited first, 30 of 57)

- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [Word-As-Image for Semantic Typography](https://mlsanity.com/paper/2303.01818/) (2023)
- [Concept Decomposition for Visual Exploration and Inspiration](https://mlsanity.com/paper/2305.18203/) (2023)
- [RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions](https://mlsanity.com/paper/2302.09466/) (2023)
- [CLIP in Medical Imaging: A Survey](https://mlsanity.com/paper/2312.07353/) (2023)
- [NOPE: Novel Object Pose Estimation from a Single Image](https://mlsanity.com/paper/2303.13612/) (2023)
- [CLIP-S4: Language-Guided Self-Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.01040/) (2023)
- [Text-to-Vector Generation with Neural Path Representation](https://mlsanity.com/paper/2405.10317/) (2024)
- [Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation](https://mlsanity.com/paper/2304.06600/) (2023)
- [Non-Contrastive Learning Meets Language-Image Pre-Training](https://mlsanity.com/paper/2210.09304/) (2022)
- [3Doodle: Compact Abstraction of Objects with 3D Strokes](https://mlsanity.com/paper/2402.03690/) (2024)
- [What Can Human Sketches Do for Object Detection?](https://mlsanity.com/paper/2303.15149/) (2023)
- [Wired Perspectives: Multi-View Wire Art Embraces Generative AI](https://mlsanity.com/paper/2311.15421/) (2023)
- [SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation](https://mlsanity.com/paper/2506.03139/) (2025)
- [Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes](https://mlsanity.com/paper/2312.04043/) (2023)
- [SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generation](https://mlsanity.com/paper/2502.08642/) (2025)
- [SVGen: Interpretable Vector Graphics Generation with Large Language Models](https://mlsanity.com/paper/2508.09168/) (2025)
- [CLIPAG: Towards Generator-Free Text-to-Image Generation](https://mlsanity.com/paper/2306.16805/) (2023)
- [Multi-Object Sketch Animation with Grouping and Motion Trajectory Priors](https://mlsanity.com/paper/2508.15535/) (2025)
- [SENS: Part-Aware Sketch-based Implicit Neural Shape Modeling](https://mlsanity.com/paper/2306.06088/) (2023)
- [FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications](https://mlsanity.com/paper/2403.06453/) (2024)
- [Freehand Sketch Generation from Mechanical Components](https://mlsanity.com/paper/2408.05966/) (2024)
- [VectorPainter: Advanced Stylized Vector Graphics Synthesis Using Stroke-Style Priors](https://mlsanity.com/paper/2405.02962/) (2024)
- [Zero-Shot Multi-Modal Artist-Controlled Retrieval and Exploration of 3D Object Sets](https://mlsanity.com/paper/2209.00682/) (2022)
- [Learning Geometry-aware Representations by Sketching](https://mlsanity.com/paper/2304.08204/) (2023)
- [Text-Guided Vector Graphics Customization](https://mlsanity.com/paper/2309.12302/) (2023)
- [Generative Escher Meshes](https://mlsanity.com/paper/2309.14564/) (2023)
- [Semi-supervised reference-based sketch extraction using a contrastive learning framework](https://mlsanity.com/paper/2407.14026/) (2024)
- [AlteredAvatar: Stylizing Dynamic 3D Avatars with Fast Style Adaptation](https://mlsanity.com/paper/2305.19245/) (2023)

## Abstract (arXiv)

Abstraction is at the heart of sketching due to the simple and minimal nature of line drawings. Abstraction entails identifying the essential visual properties of an object or scene, which requires semantic understanding and prior knowledge of high-level concepts. Abstract depictions are therefore challenging for artists, and even more so for machines. We present CLIPasso, an object sketching method that can achieve different levels of abstraction, guided by geometric and semantic simplifications. While sketch generation methods often rely on explicit sketch datasets for training, we utilize the remarkable ability of CLIP (Contrastive-Language-Image-Pretraining) to distill semantic concepts from sketches and images alike. We define a sketch as a set of B'ezier curves and use a differentiable rasterizer to optimize the parameters of the curves directly with respect to a CLIP-based perceptual loss. The abstraction degree is controlled by varying the number of strokes. The generated sketches demonstrate multiple levels of abstraction while maintaining recognizability, underlying structure, and essential visual components of the subject drawn.