mlsanityData to 7 Oct 2026

Diffusion Editing with Soft Mask: Pixel Level Redo of Image and Video with Adjustable Strength

arXiv 2610.00359 ↗First posted 30 Sep 2026cs.GR cs.AI cs.CVcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing also fits: Video generation

Abstract (arXiv)

Diffusion models with prompt and reference image-guided editing have seen rapid progress, yet they remain too coarse for pixel-level control. One promising direction is to incorporate a soft mask that specifies spatially varying edit strengths but training such fine-grained control demands expensive pixel-wise annotations, while existing zero-shot methods often yield unsatisfactory results. We introduce SoftPaint, a new zero-shot sampling method that leverages soft masks to enable a continuous spectrum of edits, from fully preserving the original content to completely re-synthesizing the masked region. Going beyond zero-shot inpainting methods, we design a Langevin-iteration-based sampler that respects per-pixel soft mask strengths, which applies universally to image and video diffusion models, enabling tasks such as video editing. The method is gradient-free, memory-efficient, and achieves smooth, pixel-level edits across multiple image and video backbones.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 35 of its 40 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

35 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  2. 2023Differential Diffusion: Giving Each Pixel Its Strength1 Jun
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  4. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov

Other references · 31

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2022Flow Matching for Generative Modelingcited by 5,426
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  12. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
19 more
  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  2. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  3. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  4. 2022Denoising Diffusion Restoration Modelscited by 951
  5. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  6. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  7. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
  8. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  9. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 229
  10. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 218
  11. 2024FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Modelscited by 214
  12. 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 168
  13. 2023Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Modelscited by 61
  14. 2024Learning Diffusion Priors from Observations by Expectation Maximizationcited by 60
  15. 2024Divide-and-Conquer Posterior Sampling for Denoising Diffusion Priorscited by 28
  16. 2023Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scalecited by 26
  17. 2023Gradpaint: Gradient-Guided Inpainting with Diffusion Modelscited by 17
  18. 2025LanPaint: Training-Free Diffusion Inpainting with Asymptotically Exact and Fast Conditional Samplingcited by 11
  19. 2026Rethinking the Diffusion Model from a Langevin Perspectivecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Streamlining Image Editing with Layered Diffusion Brushes
  2. 2024Diffusion Model-Based Image Editing: A Survey
  3. 2025MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing
  4. 2023AVID: Any-Length Video Inpainting with Diffusion Model
  5. 2026On the Controllability-Fidelity Frontier in Diffusion Editing
  6. 2023Pix2Video: Video Editing using Image Diffusion
  7. 2024Blended Latent Diffusion under Attention Control for Real-World Video Editing
  8. 2023MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers