mlsanityData to 7 Oct 2026

Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations

arXiv 2410.10792 ↗First posted 14 Oct 2024cs.LG cs.CV stat.MLcited by 162

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Generative models transform random noise into images; their inversion aims to transform images back to structured noise for recovery and editing. This paper addresses two key tasks: (i) inversion and (ii) editing of a real image using stochastic equivalents of rectified flow models (such as Flux). Although Diffusion Models (DMs) have recently dominated the field of generative modeling for images, their inversion presents faithfulness and editability challenges due to nonlinearities in drift and diffusion. Existing state-of-the-art DM inversion approaches rely on training of additional parameters or test-time optimization of latent variables; both are expensive in practice. Rectified Flows (RFs) offer a promising alternative to diffusion models, yet their inversion has been underexplored. We propose RF inversion using dynamic optimal control derived via a linear quadratic regulator. We prove that the resulting vector field is equivalent to a rectified stochastic differential equation. Additionally, we extend our framework to design a stochastic sampler for Flux. Our inversion method allows for state-of-the-art performance in zero-shot inversion and editing, outperforming prior works in stroke-to-image synthesis and semantic image editing, with large-scale human evaluations confirming user preference.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 44 of its 55 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

44 of 55 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Control27 May
  2. 2024ReNoise: Real Image Inversion Through Iterative Noising21 Mar
  3. 2023LEDITS++: Limitless Image Editing using Text-to-Image Models28 Nov
  4. 2023Effective Real Image Editing with Accelerated Iterative Diffusion Inversion10 Sep
  5. 2023Negative-prompt Inversion: Fast Image Inversion for Editing with Text-guided Diffusion Models26 May
  6. 2023An Edit Friendly DDPM Noise Space: Inversion and Manipulations12 Apr
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  8. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Models17 Nov
  9. 2022DiffEdit: Diffusion-based semantic image editing with mask guidance20 Oct
  10. 2022Diffusion-based Image Translation using Disentangled Style and Content Representation30 Sep
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  12. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
2 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  2. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation6 Oct

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2022Flow Matching for Generative Modelingcited by 5,426
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
18 more
  1. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  3. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  5. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  6. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  7. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  8. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  9. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  10. 2023Style Aligned Image Generation via Shared Attentioncited by 214
  11. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  12. 2023Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restorationcited by 164
  13. 2023HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generationcited by 104
  14. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  15. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusioncited by 78
  16. 2023A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Modelscited by 37
  17. 2016Knowledge Guided Disambiguation for Large-Scale Scene Classification with Multi-Resolution CNNscited by 16
  18. 2020Lecture Notes on Control System Theory and Designcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 79; 2026: 68.

162 papers.

Sort:

By area:

  1. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  2. 2024FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Modelscited by 214
  3. 2024Taming Rectified Flow for Inversion and Editingcited by 200
  4. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  5. 2024FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editingcited by 66
  6. 2025KV-Edit: Training-Free Image Editing for Precise Background Preservationcited by 47
  7. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  8. 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 40
  9. 2024Steering Rectified Flow Models in the Vector Field for Controlled Image Generationcited by 38
  10. 2025UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Modelscited by 38
  11. 2024LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relightingcited by 37
  12. 2024AnimateAnything: Consistent and Controllable Animation for Video Generationcited by 34
38 more
  1. 2024Unveil Inversion and Invariance in Flow Transformer for Versatile Image Editingcited by 34
  2. 2025FlowAlign: Trajectory-Regularized, Inversion-Free Flow-based Image Editingcited by 31
  3. 2025DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editingcited by 29
  4. 2025Follow-Your-Shape: Shape-Aware Image Editing via Trajectory-Guided Region Controlcited by 27
  5. 2025UniCombine: Unified Multi-Conditional Combination with Diffusion Transformercited by 25
  6. 2025Personalize Anything for Free with Diffusion Transformercited by 22
  7. 2025DNAEdit: Direct Noise Alignment for Text-Guided Rectified Flow Editingcited by 22
  8. 2025Training-Free Text-Guided Image Editing with Visual Autoregressive Modelcited by 21
  9. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  10. 2024Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editingcited by 19
  11. 2025Multi-turn Consistent Image Editingcited by 17
  12. 2025Exploring Multimodal Diffusion Transformers for Enhanced Prompt-based Image Editingcited by 16
  13. 2025FiVE: A Fine-grained Video Editing Benchmark for Evaluating Emerging Diffusion and Rectified Flow Modelscited by 15
  14. 2025ReFlex: Text-Guided Editing of Real Images in Rectified Flow via Mid-Step Feature Extraction and Attention Adaptationcited by 15
  15. 2025SplitFlow: Flow Decomposition for Inversion-Free Text-to-Image Editingcited by 15
  16. 2025EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editingcited by 12
  17. 2025Flux Already Knows -- Activating Subject-Driven Image Generation without Trainingcited by 11
  18. 2025Adams Bashforth Moulton Solver for Inversion and Editing in Rectified Flowcited by 10
  19. 2025Discrete-Guided Diffusion for Scalable and Safe Multi-Robot Motion Planningcited by 10
  20. 2024Memories of Forgotten Conceptscited by 9
  21. 2024Edicho: Consistent Image Editing in the Wildcited by 9
  22. 2025PairEdit: Learning Semantic Variations for Exemplar-based Image Editingcited by 8
  23. 2025SAEdit: Token-level control for continuous image editing via Sparse AutoEncodercited by 8
  24. 2025Group Relative Attention Guidance for Image Editingcited by 8
  25. 2025Tight Inversion: Image-Conditioned Inversion for Real Image Editingcited by 7
  26. 2025UniEdit-I: Training-free Image Editing for Unified VLM via Iterative Understanding, Editing and Verifyingcited by 7
  27. 2025Noise Matters: Optimizing Matching Noise for Diffusion Classifierscited by 7
  28. 2024Constrained Posterior Sampling: Time Series Generation with Hard Constraintscited by 6
  29. 2025TF-TI2I: Training-Free Text-and-Image-to-Image Generation via Multi-Modal Implicit-Context Learning in Text-to-Image Modelscited by 6
  30. 2025Product of Experts for Visual Generationcited by 6
  31. 2025Describe, Don't Dictate: Semantic Image Editing with Natural Language Intentcited by 6
  32. 2025Training-Free Multi-Style Fusion Through Reference-Based Adaptive Modulationcited by 6
  33. 2025Exploring Cross-Modal Flows for Few-Shot Learningcited by 6
  34. 2025ConsistEdit: Highly Consistent and Precise Training-free Visual Editingcited by 6
  35. 2025Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimizationcited by 6
  36. 2024MyTimeMachine: Personalized Facial Age Transformationcited by 5
  37. 2025Optimal Transport for Rectified Flow Image Editing: Unifying Inversion-Based and Direct Methodscited by 5
  38. 2025Training-Free Text-Guided Color Editing with Multi-Modal Diffusion Transformercited by 5

50 most cited of 162 All 162 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023High-Fidelity Diffusion-based Image Editing
  2. 2024Text-to-Image Rectified Flow as Plug-and-Play Priors
  3. 2025DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing
  4. 2024TurboEdit: Instant text-based image editing
  5. 2023Inversion-Free Image Editing with Natural Language
  6. 2024Eta Inversion: Designing an Optimal Eta Function for Diffusion-based Real Image Editing
  7. 2025Solving Inverse Problems with FLAIR
  8. 2026Training-free image inversion for one-step diffusion models