mlsanityData to 7 Oct 2026

CLIPascene: Scene Sketching with Different Types and Levels of Abstraction

arXiv 2211.17256 ↗First posted 30 Nov 2022cs.CV cs.GRcited by 63

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a method for converting a given scene image into a sketch using different types and multiple levels of abstraction. We distinguish between two types of abstraction. The first considers the fidelity of the sketch, varying its representation from a more precise portrayal of the input to a looser depiction. The second is defined by the visual simplicity of the sketch, moving from a detailed depiction to a sparse sketch. Using an explicit disentanglement into two abstraction axes -- and multiple levels for each one -- provides users additional control over selecting the desired sketch based on their personal goals and preferences. To form a sketch at a given level of fidelity and simplification, we train two MLP networks. The first network learns the desired placement of strokes, while the second network learns to gradually remove strokes from the sketch without harming its recognizability and semantics. Our approach is able to generate sketches of complex scenes including those with complex backgrounds (e.g., natural and urban settings) and subjects (e.g., animals and people) while depicting gradual abstractions of the input scene in terms of fidelity and simplicity.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 30 Nov 2022.

Builds on

36 of 54 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
24 more
  1. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  2. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  3. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  4. 2017Self-Normalizing Neural Networkscited by 591
  5. 2015Holistically-Nested Edge Detectioncited by 590
  6. 2020U2-Net: Going Deeper with Nested U-Structure for Salient Object Detectioncited by 410
  7. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
  8. 2017A Neural Representation of Sketch Drawingscited by 311
  9. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  10. 2022Learning to generate line drawings that convey geometry and semanticscited by 82
  11. 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 67
  12. 2020Deep Learning for Free-Hand Sketch: A Surveycited by 59
  13. 2018Learning to Sketch with Shortcut Cycle Consistencycited by 47
  14. 2020Creative Sketch Generationcited by 44
  15. 2018Learning Deep Sketch Abstractioncited by 36
  16. 2020Why Do Line Drawings Work? A Realism Hypothesiscited by 28
  17. 2021Learning to Draw: Emergent Communication through Sketchingcited by 23
  18. 2021SketchLattice: Latticed Representation for Sketch Manipulationcited by 19
  19. 2019Im2Pencil: Controllable Pencil Illustration from Photographscited by 18
  20. 2021DoodleFormer: Creative Sketch Drawing with Transformerscited by 17
  21. 2015Free-hand Sketch Synthesis with Deformable Stroke Modelscited by 16
  22. 2021Emergent Graphical Conventions in a Visual Communication Gamecited by 16
  23. 2021Neural Strokes: Stylized Line Drawing of 3D Shapescited by 13
  24. 2020Sketch Generation with Drawing Process Guided by Vector Flow and Grayscalecited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 12; 2025: 23; 2026: 11.

63 papers.

Sort:

By area:

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  2. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  3. 2023SVGDreamer: Text Guided SVG Generation with Diffusion Modelcited by 75
  4. 2023Word-As-Image for Semantic Typographycited by 62
  5. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61
  6. 2023DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Modelscited by 54
  7. 2023Breathing Life Into Sketches Using Text-to-Video Priorscited by 47
  8. 2024RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketchescited by 41
  9. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
  10. 2023ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracycited by 19
  11. 2025NeuralSVG: An Implicit Representation for Text-to-Vector Generationcited by 16
  12. 2025SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generationcited by 14
38 more
  1. 2024SketchINR: A First Look into Sketches as Implicit Neural Representationscited by 13
  2. 2026Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedbackcited by 12
  3. 2024Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Renderingcited by 9
  4. 2025Multi-Object Sketch Animation with Grouping and Motion Trajectory Priorscited by 9
  5. 2024FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animationscited by 8
  6. 2025Image Editing with Diffusion Models: A Surveycited by 8
  7. 2024VectorPainter: Advanced Stylized Vector Graphics Synthesis Using Stroke-Style Priorscited by 7
  8. 2023Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representationscited by 5
  9. 2024LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Modelcited by 5
  10. 2025ViewCraft3D: High-Fidelity and View-Consistent 3D Vector Graphics Synthesiscited by 5
  11. 2023Sketch Video Synthesiscited by 4
  12. 2023Open Vocabulary Semantic Scene Sketch Understandingcited by 4
  13. 2023Fine-Tuned but Zero-Shot 3D Shape Sketch View Similarity and Retrievalcited by 3
  14. 2023Emergent Communication in Interactive Sketch Question Answeringcited by 3
  15. 2024CraftSVG: Multi-Object Text-to-SVG Synthesis via Layout Guided Diffusioncited by 3
  16. 2025MixSA: Training-free Reference-based Sketch Extraction via Mixture-of-Self-Attentioncited by 3
  17. 2025WIR3D: Visually-Informed and Geometry-Aware 3D Shape Abstractioncited by 3
  18. 2025Neural Image Abstraction Using Long Smoothing B-Splinescited by 3
  19. 2026Deep Sketch-Based 3D Modeling: A Surveycited by 3
  20. 2026AmodalSVG: Amodal Image Vectorization via Semantic Layer Peelingcited by 3
  21. 2024Cross-Modal Few-Shot Learning: a Generative Transfer Learning Frameworkcited by 2
  22. 2025AutoSketch: VLM-assisted Style-Aware Vector Sketch Completioncited by 2
  23. 2025Recovering Dynamic 3D Sketches from Videoscited by 2
  24. 2025StrokeFusion: Vector Sketch Generation via Joint Stroke-UDF Encoding and Latent Sequence Diffusioncited by 2
  25. 2025CoProSketch: Controllable and Progressive Sketch Generation with Diffusion Modelcited by 2
  26. 2026VideoSketcher: Sequential Sketch Generation Using Video Model Priorscited by 2
  27. 2024SketchRef: a Multi-Task Evaluation Benchmark for Sketch Synthesiscited by 1
  28. 2025Sketch & Paint: Stroke-by-Stroke Evolution of Visual Artworkscited by 1
  29. 2025IlluSign: Illustrating Sign Language Videos by Leveraging the Attention Mechanismcited by 1
  30. 2025Empowering Vector Graphics with Consistently Arbitrary Viewing and View-dependent Visibilitycited by 1
  31. 2025Stroke2Sketch: Harnessing Stroke Attributes for Training-Free Sketch Generationcited by 1
  32. 2026Stroke of Surprise: Progressive Semantic Illusions in Vector Sketchingcited by 1
  33. 2026Teaching an Agent to Sketch One Part at a Timecited by 1
  34. 2023Sketch2CAD: 3D CAD Model Reconstruction from 2D Sketch using Visual Transformercited by 0
  35. 2023Adversarial Doodles: Interpretable and Human-drawable Attacks Provide Describable Insightscited by 0
  36. 2023CLIPDraw++: Text-to-Sketch Synthesis with Simple Primitivescited by 0
  37. 2023Optimize and Reduce: A Top-Down Approach for Image Vectorizationcited by 0
  38. 2024Scene Depth Estimation from Traditional Oriental Landscape Paintingscited by 0

50 most cited of 63 All 63 (JSON)