# Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation

arXiv: https://arxiv.org/abs/1909.11081 · HTML: https://mlsanity.com/paper/1909.11081/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2019.

## Built on by (most cited first, 30 of 51)

- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Sketch-Guided Text-to-Image Diffusion Models](https://mlsanity.com/paper/2211.13752/) (2022)
- [Multimodal Conditional Image Synthesis with Product-of-Experts GANs](https://mlsanity.com/paper/2112.05130/) (2021)
- [3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows](https://mlsanity.com/paper/2210.11603/) (2022)
- [M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers](https://mlsanity.com/paper/2105.14211/) (2021)
- [IMAGGarment: Fine-Grained Garment Generation for Controllable Fashion Design](https://mlsanity.com/paper/2504.13176/) (2025)
- [Picture that Sketch: Photorealistic Image Generation from Abstract Sketches](https://mlsanity.com/paper/2303.11162/) (2023)
- [SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches](https://mlsanity.com/paper/2111.15078/) (2021)
- [Probabilistic Compositional Embeddings for Multimodal Image Retrieval](https://mlsanity.com/paper/2204.05845/) (2022)
- [It's All About Your Sketch: Democratising Sketch Control in Diffusion Models](https://mlsanity.com/paper/2403.07234/) (2024)
- [Sketching without Worrying: Noise-Tolerant Sketch-Based Image Retrieval](https://mlsanity.com/paper/2203.14817/) (2022)
- [MaskSketch: Unpaired Structure-guided Masked Image Generation](https://mlsanity.com/paper/2302.05496/) (2023)
- [Towards Open-World Text-Guided Face Image Generation and Manipulation](https://mlsanity.com/paper/2104.08910/) (2021)
- [AI-Generated Content (AIGC) for Various Data Modalities: A Survey](https://mlsanity.com/paper/2308.14177/) (2023)
- [Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis](https://mlsanity.com/paper/2104.05703/) (2021)
- [ScribbleLight: Single Image Indoor Relighting with Scribbles](https://mlsanity.com/paper/2411.17696/) (2024)
- [Partially Does It: Towards Scene-Level FG-SBIR with Partial Input](https://mlsanity.com/paper/2203.14804/) (2022)
- [SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Control](https://mlsanity.com/paper/2312.05039/) (2023)
- [Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings](https://mlsanity.com/paper/2303.11502/) (2023)
- [CoGS: Controllable Generation and Search from Sketch and Style](https://mlsanity.com/paper/2203.09554/) (2022)
- [Facial-Sketch Synthesis: A New Challenge](https://mlsanity.com/paper/2112.15439/) (2021)
- [DiffSketching: Sketch Control Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2305.18812/) (2023)
- [Conditional Temporal Variational AutoEncoder for Action Video Prediction](https://mlsanity.com/paper/2108.05658/) (2021)
- [One Sketch for All: One-Shot Personalized Sketch Segmentation](https://mlsanity.com/paper/2112.10838/) (2021)
- [dualFace:Two-Stage Drawing Guidance for Freehand Portrait Sketching](https://mlsanity.com/paper/2104.12297/) (2021)
- [Deep Image Synthesis from Intuitive User Input: A Review and Perspectives](https://mlsanity.com/paper/2107.04240/) (2021)
- [Image Search with Text Feedback by Additive Attention Compositional Learning](https://mlsanity.com/paper/2203.03809/) (2022)
- [AniFaceDrawing: Anime Portrait Exploration during Your Sketching](https://mlsanity.com/paper/2306.07476/) (2023)
- [LOTS of Fashion! Multi-Conditioning for Image Generation via Sketch-Text Pairing](https://mlsanity.com/paper/2507.22627/) (2025)
- [Paint2Pix: Interactive Painting based Progressive Image Synthesis and Editing](https://mlsanity.com/paper/2208.08092/) (2022)

## Abstract (arXiv)

We propose an interactive GAN-based sketch-to-image translation method that helps novice users create images of simple objects. As the user starts to draw a sketch of a desired object type, the network interactively recommends plausible completions, and shows a corresponding synthesized image to the user. This enables a feedback loop, where the user can edit their sketch based on the network's recommendations, visualizing both the completed shape and final rendered image while they draw. In order to use a single trained model across a wide array of object classes, we introduce a gating-based approach for class conditioning, which allows us to generate distinct classes without feature mixing, from a single generator network. Video available at our website: https://arnabgho.github.io/iSketchNFill/.