mlsanityData to 13 Oct 2026

ControlFill: Spatially Adjustable Image Inpainting from Prompt Learning

arXiv 2503.04268 ↗First posted 6 Mar 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In this report, I present an inpainting framework named textitControlFill, which involves training two distinct prompts: one for generating plausible objects within a designated mask (textitcreation) and another for filling the region by extending the background (textitremoval). During the inference stage, these learned embeddings guide a diffusion network that operates without requiring heavy text encoders. By adjusting the relative significance of the two prompts and employing classifier-free guidance, users can control the intensity of removal or creation. Furthermore, I introduce a method to spatially vary the intensity of guidance by assigning different scales to individual pixels.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 24 of its 28 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

24 of 28 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,231
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,082
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,213
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,564
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,339
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,323
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,294
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,407
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,240
12 more
  1. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,490
  2. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,425
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,298
  4. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,159
  5. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 871
  6. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 733
  7. 2022Blended Latent Diffusioncited by 432
  8. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 285
  9. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
  10. 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 168
  11. 2022NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesiscited by 71
  12. 2024Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidancecited by 58

Built on by

Not cited yet.