mlsanityData to 7 Oct 2026

Character Generation through Self-Supervised Vectorization

arXiv 2208.02012 ↗First posted 3 Aug 2022cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The prevalent approach in self-supervised image generation is to operate on pixel level representations. While this approach can produce high quality images, it cannot benefit from the simplicity and innate quality of vectorization. Here we present a drawing agent that operates on stroke-level representation of images. At each time step, the agent first assesses the current canvas and decides whether to stop or keep drawing. When a 'draw' decision is made, the agent outputs a program indicating the stroke to be drawn. As a result, it produces a final raster image by drawing the strokes on a canvas, using a minimal number of strokes and dynamically deciding when to stop. We train our agent through reinforcement learning on MNIST and Omniglot datasets for unconditional generation and parsing (reconstruction) tasks. We utilize our parsing agent for exemplar generation and type conditioned concept generation in Omniglot challenge without any further training. We present successful results on all three generation tasks and the parsing task. Crucially, we do not need any stroke-level or vector supervision; we only use raster images for training.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 3 Aug 2022.

Builds on

42 of 55 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Generative Adversarial Networkscited by 5,603
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  8. 2017Improved Training of Wasserstein GANscited by 2,223
  9. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  10. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  11. 2016Least Squares Generative Adversarial Networkscited by 1,000
  12. 2016Pixel Recurrent Neural Networkscited by 814
30 more
  1. 2015A Neural Algorithm of Artistic Stylecited by 480
  2. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  3. 2017A Neural Representation of Sketch Drawingscited by 311
  4. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  5. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  6. 2016Towards a Neural Statisticiancited by 119
  7. 2021Im2Vec: Synthesizing Vector Graphics without Vector Supervisioncited by 114
  8. 2017Building machines that adapt and compute like brainscited by 99
  9. 2019Write, Execute, Assess: Program Synthesis with a REPLcited by 92
  10. 2020Sketchformer: Transformer-based Representation for Sketched Structurecited by 78
  11. 2018Synthesizing Programs for Images using Reinforced Adversarial Learningcited by 72
  12. 2019Learning to Paint With Model-based Deep Reinforcement Learningcited by 70
  13. 2020Stylized Neural Paintingcited by 50
  14. 2016One-Shot Generalization in Deep Generative Modelscited by 44
  15. 2020B'ezierSketch: A generative model for scalable vector sketchescited by 43
  16. 2020Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestaltcited by 42
  17. 2021Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwritingcited by 38
  18. 2016Towards Conceptual Compressioncited by 36
  19. 2019Neural Painters: A learned differentiable constraint for generating brushstroke paintingscited by 28
  20. 2020CoSE: Compositional Stroke Embeddingscited by 26
  21. 2018The Variational Homoencoder: Learning to learn high capacity generative models from few examplescited by 23
  22. 2019Unsupervised Doodling and Painting with Improved SPIRALcited by 21
  23. 2017Sketch-pix2seq: a Model to Generate Sketches of Multiple Categoriescited by 20
  24. 2020SketchEmbedNet: Learning Novel Concepts by Imitating Drawingscited by 18
  25. 2021Cloud2Curve: Generation and Vectorization of Parametric Sketchescited by 16
  26. 2020Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agentcited by 15
  27. 2020Learning Task-General Representations with Generative Neuro-Symbolic Modelingcited by 14
  28. 2020Generating new concepts with hybrid neuro-symbolic modelscited by 9
  29. 2018Learning to Sketch with Deep Q Networks and Demonstrated Strokescited by 6
  30. 2017High-Quality Face Image SR Using Conditional Generative Adversarial Networkscited by 4

Built on by

Not cited yet.