# DiffUHaul: A Training-Free Method for Object Dragging in Images

arXiv: https://arxiv.org/abs/2406.01594 · HTML: https://mlsanity.com/paper/2406.01594/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 47 of its 72 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 47)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [FateZero: Fusing Attentions for Zero-shot Text-based Video Editing](https://mlsanity.com/paper/2303.09535/) (2023)
- [TokenFlow: Consistent Diffusion Features for Consistent Video Editing](https://mlsanity.com/paper/2307.10373/) (2023)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [Training-Free Layout Control with Cross-Attention Guidance](https://mlsanity.com/paper/2304.03373/) (2023)
- [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023)
- [LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation](https://mlsanity.com/paper/2303.17189/) (2023)
- [An Edit Friendly DDPM Noise Space: Inversion and Manipulations](https://mlsanity.com/paper/2304.06140/) (2023)
- [Break-A-Scene: Extracting Multiple Concepts from a Single Image](https://mlsanity.com/paper/2305.16311/) (2023)

## Built on by (most cited first, 22 of 22)

- [Stable Flow: Vital Layers for Training-Free Image Editing](https://mlsanity.com/paper/2411.14430/) (2024)
- [Add-it: Training-Free Object Insertion in Images With Pretrained Diffusion Models](https://mlsanity.com/paper/2411.07232/) (2024)
- [ObjectMover: Generative Object Movement with Video Prior](https://mlsanity.com/paper/2503.08037/) (2025)
- [PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Models](https://mlsanity.com/paper/2503.10127/) (2025)
- [Story2Board: A Training-Free Approach for Expressive Storyboard Generation](https://mlsanity.com/paper/2508.09983/) (2025)
- [VideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priors](https://mlsanity.com/paper/2503.01107/) (2025)
- [Motion Modes: What Could Happen Next?](https://mlsanity.com/paper/2412.00148/) (2024)
- [3D-Fixup: Advancing Photo Editing with 3D Priors](https://mlsanity.com/paper/2505.10566/) (2025)
- [Compass Control: Multi Object Orientation Control for Text-to-Image Generation](https://mlsanity.com/paper/2504.06752/) (2025)
- [Tight Inversion: Image-Conditioned Inversion for Real Image Editing](https://mlsanity.com/paper/2502.20376/) (2025)
- [A Diffusion-Based Framework for Occluded Object Movement](https://mlsanity.com/paper/2504.01873/) (2025)
- [Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentions](https://mlsanity.com/paper/2407.19205/) (2024)
- [Motion by Queries: Identity-Motion Trade-offs in Text-to-Video Generation](https://mlsanity.com/paper/2412.07750/) (2024)
- [FreeCus: Free Lunch Subject-driven Customization in Diffusion Transformers](https://mlsanity.com/paper/2507.15249/) (2025)
- [BlobCtrl: Taming Controllable Blob for Element-level Image Editing](https://mlsanity.com/paper/2503.13434/) (2025)
- [Controllable 3D Placement of Objects with Scene-Aware Diffusion Models](https://mlsanity.com/paper/2506.21446/) (2025)
- [Understanding the Implicit User Intention via Reasoning with Large Language Model for Image Editing](https://mlsanity.com/paper/2510.27335/) (2025)
- [DynaDrag: Dynamic Drag-Style Image Editing by Motion Prediction](https://mlsanity.com/paper/2601.00542/) (2026)
- [LightMover: Generative Light Movement with Color and Intensity Controls](https://mlsanity.com/paper/2603.27209/) (2026)
- [AdaEraser: Training-Free Object Removal via Adaptive Attention Suppression](https://mlsanity.com/paper/2605.15921/) (2026)
- [Phi-Noise: Training-Free Temporal Video Conditioning via Phase-Based Noise Manipulation](https://mlsanity.com/paper/2605.24509/) (2026)
- [In-context Region-based Drag: Drag Any Region to Any Shape](https://mlsanity.com/paper/2606.25907/) (2026)

## Abstract (arXiv)

Text-to-image diffusion models have proven effective for solving many image editing tasks. However, the seemingly straightforward task of seamlessly relocating objects within a scene remains surprisingly challenging. Existing methods addressing this problem often struggle to function reliably in real-world scenarios due to lacking spatial reasoning. In this work, we propose a training-free method, dubbed DiffUHaul, that harnesses the spatial understanding of a localized text-to-image model, for the object dragging task. Blindly manipulating layout inputs of the localized model tends to cause low editing performance due to the intrinsic entanglement of object representation in the model. To this end, we first apply attention masking in each denoising step to make the generation more disentangled across different objects and adopt the self-attention sharing mechanism to preserve the high-level object appearance. Furthermore, we propose a new diffusion anchoring technique: in the early denoising steps, we interpolate the attention features between source and target images to smoothly fuse new layouts with the original appearance; in the later denoising steps, we pass the localized features from the source images to the interpolated images to retain fine-grained object details. To adapt DiffUHaul to real-image editing, we apply a DDPM self-attention bucketing that can better reconstruct real images with the localized model. Finally, we introduce an automated evaluation pipeline for this task and showcase the efficacy of our method. Our results are reinforced through a user preference study.