mlsanityData to 7 Oct 2026

Sketch Video Synthesis

arXiv 2311.15306 ↗First posted 26 Nov 2023cs.CV cs.GRcited by 4

Not on the map. Why?

Abstract (arXiv)

Understanding semantic intricacies and high-level concepts is essential in image sketch generation, and this challenge becomes even more formidable when applied to the domain of videos. To address this, we propose a novel optimization-based framework for sketching videos represented by the frame-wise B'ezier curve. In detail, we first propose a cross-frame stroke initialization approach to warm up the location and the width of each curve. Then, we optimize the locations of these curves by utilizing a semantic loss based on CLIP features and a newly designed consistency loss using the self-decomposed 2D atlas network. Built upon these design elements, the resulting sketch video showcases impressive visual abstraction and temporal coherence. Furthermore, by transforming a video into SVG lines through the sketching process, our method unlocks applications in sketch-based video editing and video doodling, enabled through video composition, as exemplified in the teaser.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 26 Nov 2023.

Builds on

34 of 49 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  9. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  10. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  11. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  12. 2015Holistically-Nested Edge Detectioncited by 590
22 more
  1. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  2. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  3. 2018Video-to-Video Synthesiscited by 388
  4. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  5. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  6. 2017A Neural Representation of Sketch Drawingscited by 311
  7. 2018Learning Blind Video Temporal Consistencycited by 247
  8. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  9. 2020DeepSVG: A Hierarchical Generative Network for Vector Graphics Animationcited by 158
  10. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  11. 2021Robust High-Resolution Video Matting with Temporal Guidancecited by 136
  12. 2021Im2Vec: Synthesizing Vector Graphics without Vector Supervisioncited by 114
  13. 2020Blind Video Temporal Consistency via Deep Video Priorcited by 76
  14. 2023IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformerscited by 73
  15. 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 67
  16. 2022CLIPascene: Scene Sketching with Different Types and Levels of Abstractioncited by 63
  17. 2023Word-As-Image for Semantic Typographycited by 62
  18. 2020Deep Learning for Free-Hand Sketch: A Surveycited by 59
  19. 2020B'ezierSketch: A generative model for scalable vector sketchescited by 43
  20. 2023Blind Video Deflickering by Neural Filtering with a Flawed Atlascited by 27
  21. 2022Deep Video Prior for Video Consistency and Propagationcited by 18
  22. 2020Synthesizing human-like sketches from natural images using a conditional convolutional decodercited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animationscited by 8
  2. 2025Recovering Dynamic 3D Sketches from Videoscited by 2
  3. 2025Vector sketch animation generation with differentiable motion trajectoriescited by 0
  4. 2026DrawVideo: Grounded and Faithful Multi-Shot Video Generation from Storyboard Keyframe Sketchescited by 0