# SketchLattice: Latticed Representation for Sketch Manipulation arXiv: https://arxiv.org/abs/2108.11636 ยท HTML: https://mlsanity.com/paper/2108.11636/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 35 references that are in the map. - First posted on arXiv 26 Aug 2021. ## Builds on (most cited first, 20 of 20) - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018) - [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013) - [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019) - [A Neural Representation of Sketch Drawings](https://mlsanity.com/paper/1704.03477/) (2017) - [Graph R-CNN for Scene Graph Generation](https://mlsanity.com/paper/1808.00191/) (2018) - [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018) - [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019) - [Exploring Visual Relationship for Image Captioning](https://mlsanity.com/paper/1809.07041/) (2018) - [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020) - [Scribbler: Controlling Deep Image Synthesis with Sketch and Color](https://mlsanity.com/paper/1612.00835/) (2016) - [Disentangled Representation Learning for 3D Face Shape](https://mlsanity.com/paper/1902.09887/) (2019) - [Learning to Sketch with Shortcut Cycle Consistency](https://mlsanity.com/paper/1805.00247/) (2018) - [3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks](https://mlsanity.com/paper/1707.06375/) (2017) - [Unsupervised Sketch-to-Photo Synthesis](https://mlsanity.com/paper/1909.08313/) (2019) - [Learning Deep Sketch Abstraction](https://mlsanity.com/paper/1804.04804/) (2018) - [Multi-Graph Transformer for Free-Hand Sketch Recognition](https://mlsanity.com/paper/1912.11258/) (2019) - [Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories](https://mlsanity.com/paper/1709.04121/) (2017) - [3D Shape Reconstruction from Free-Hand Sketches](https://mlsanity.com/paper/2006.09694/) (2020) - [PointRGCN: Graph Convolution Networks for 3D Vehicles Detection Refinement](https://mlsanity.com/paper/1911.12236/) (2019) - [SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks](https://mlsanity.com/paper/2003.00678/) (2020) ## Built on by (most cited first, 19 of 19) - [CLIPascene: Scene Sketching with Different Types and Levels of Abstraction](https://mlsanity.com/paper/2211.17256/) (2022) - [CLIPasso: Semantically-Aware Object Sketching](https://mlsanity.com/paper/2202.05822/) (2022) - [DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models](https://mlsanity.com/paper/2306.14685/) (2023) - [FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context](https://mlsanity.com/paper/2203.02113/) (2022) - [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024) - [Partially Does It: Towards Scene-Level FG-SBIR with Partial Input](https://mlsanity.com/paper/2203.14804/) (2022) - [DoodleFormer: Creative Sketch Drawing with Transformers](https://mlsanity.com/paper/2112.03258/) (2021) - [Semi-supervised reference-based sketch extraction using a contrastive learning framework](https://mlsanity.com/paper/2407.14026/) (2024) - [Linking Sketch Patches by Learning Synonymous Proximity for Graphic Sketch Representation](https://mlsanity.com/paper/2211.16841/) (2022) - [WIR3D: Visually-Informed and Geometry-Aware 3D Shape Abstraction](https://mlsanity.com/paper/2505.04813/) (2025) - [Equipping Sketch Patches with Context-Aware Positional Encoding for Graphic Sketch Representation](https://mlsanity.com/paper/2403.17525/) (2024) - [Do Generalised Classifiers really work on Human Drawn Sketches?](https://mlsanity.com/paper/2407.03893/) (2024) - [VideoSketcher: Sequential Sketch Generation Using Video Model Priors](https://mlsanity.com/paper/2602.15819/) (2026) - [Conditional Human Sketch Synthesis with Explicit Abstraction Control](https://mlsanity.com/paper/2306.09274/) (2023) - [Sketch & Paint: Stroke-by-Stroke Evolution of Visual Artworks](https://mlsanity.com/paper/2502.20119/) (2025) - [SketchBodyNet: A Sketch-Driven Multi-faceted Decoder Network for 3D Human Reconstruction](https://mlsanity.com/paper/2310.06577/) (2023) - [Generative Visual Communication in the Era of Vision-Language Models](https://mlsanity.com/paper/2411.18727/) (2024) - [Text to Sketch Generation with Multi-Styles](https://mlsanity.com/paper/2511.04123/) (2025) - [SketchMamba: A Lightweight State-Space Model for Joint Progressive Sketch Classification and Stroke Auto-Completion](https://mlsanity.com/paper/2607.23580/) (2026) ## Abstract (arXiv) The key challenge in designing a sketch representation lies with handling the abstract and iconic nature of sketches. Existing work predominantly utilizes either, (i) a pixelative format that treats sketches as natural images employing off-the-shelf CNN-based networks, or (ii) an elaborately designed vector format that leverages the structural information of drawing orders using sequential RNN-based methods. While the pixelative format lacks intuitive exploitation of structural cues, sketches in vector format are absent in most cases limiting their practical usage. Hence, in this paper, we propose a lattice structured sketch representation that not only removes the bottleneck of requiring vector data but also preserves the structural cues that vector data provides. Essentially, sketch lattice is a set of points sampled from the pixelative format of the sketch using a lattice graph. We show that our lattice structure is particularly amenable to structural changes that largely benefits sketch abstraction modeling for generation tasks. Our lattice representation could be effectively encoded using a graph model, that uses significantly fewer model parameters (13.5 times lesser) than existing state-of-the-art. Extensive experiments demonstrate the effectiveness of sketch lattice for sketch manipulation, including sketch healing and image-to-sketch synthesis.