EVA: Zero-shot Accurate Attributes and Multi-Object Video Editing
On the map: Diffusion models → Video generation
Abstract (arXiv)
Current diffusion-based video editing primarily focuses on local editing (textite.g., object/background editing) or global style editing by utilizing various dense correspondences. However, these methods often fail to accurately edit the foreground and background simultaneously while preserving the original layout. We find that the crux of the issue stems from the imprecise distribution of attention weights across designated regions, including inaccurate text-to-attribute control and attention leakage. To tackle this issue, we introduce EVA, a zero-shot and multi-attribute video editing framework tailored for human-centric videos with complex motions. We incorporate a Spatial-Temporal Layout-Guided Attention mechanism that leverages the intrinsic positive and negative correspondences of cross-frame diffusion features. To avoid attention leakage, we utilize these correspondences to boost the attention scores of tokens within the same attribute across all video frames while limiting interactions between tokens of different attributes in the self-attention layer. For precise text-to-attribute manipulation, we use discrete text embeddings focused on specific layout areas within the cross-attention layer. Benefiting from the precise attention weight distribution, EVA can be easily generalized to multi-object editing scenarios and achieves accurate identity mapping. Extensive experiments demonstrate EVA achieves state-of-the-art results in real-world scenarios. Full results are provided at https://knightyxp.github.io/EVA/
Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 40 of its 43 references that are in the map. First posted on arXiv 24 Mar 2024.
Builds on
40 of 43 references.
Closest ancestors · 13
Same direction, newest first
- 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondence4 Dec
- 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Models12 Oct
- 2023FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editing9 Oct
- 2023Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models2 Oct
- 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing18 Aug
- 2023ModelScope Text-to-Video Technical Report12 Aug
- 2023VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet26 Jul
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
- 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation13 Jun
- 2023ControlVideo: Training-free Controllable Text-to-Video Generation22 May
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editing16 Mar
1 more
Other references · 27
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
- 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
- 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
15 more
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
- 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
- 2023Zero-shot Image-to-Image Translationcited by 503
- 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
- 2023Emergent Correspondence from Image Diffusioncited by 437
- 2022Blended Latent Diffusioncited by 430
- 2022MAGVIT: Masked Generative Video Transformercited by 341
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
- 2018Learning Blind Video Temporal Consistencycited by 247
- 2023Segment and Track Anythingcited by 193
- 2023Grounded Text-to-Image Synthesis with Attention Refocusingcited by 163
- 2023Dense Text-to-Image Generation with Attention Modulationcited by 162
- 2021Layered Neural Atlases for Consistent Video Editingcited by 137
- 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
- 2023Dynamic Prompt Learning: Addressing Cross-Attention Leakage for Text-Based Image Editingcited by 76
Built on by
Citing papers per year: 2024: 6; 2025: 8; 2026: 3.
17 papers.
Sort:
By area:
- 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
- 2024Dual-Personalizing Adapter for Federated Foundation Modelscited by 54
- 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editingcited by 50
- 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Controlcited by 42
- 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
- 2025FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusioncited by 15
- 2025ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfercited by 12
- 2024EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Modelscited by 10
- 2026Evolution of Video Generative Foundationscited by 6
- 2025VideoCoF: Unified Video Editing with Temporal Reasonercited by 5
- 2026ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editingcited by 3
- 2024MAKIMA: Tuning-free Multi-Attribute Open-domain Video Editing via Mask-Guided Attention Modulationcited by 2
5 more
- 2025Learning to Animate Images from A Few Videos to Portray Delicate Human Actionscited by 1
- 2024PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Datasetcited by 0
- 2025MOVi: Training-free Text-conditioned Multi-Object Video Generationcited by 0
- 2025VIDMP3: Video Editing by Representing Motion with Pose and Position Priorscited by 0
- 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomycited by 0
- 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy28 Sep 2026
- 2026ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing23 Jul 2026
- 2026Evolution of Video Generative Foundations7 Apr 2026
- 2025VideoCoF: Unified Video Editing with Temporal Reasoner8 Dec 2025
- 2025VIDMP3: Video Editing by Representing Motion with Pose and Position Priors14 Oct 2025
- 2025FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusion30 Jun 2025
- 2025MOVi: Training-free Text-conditioned Multi-Object Video Generation29 May 2025
- 2025ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfer3 Apr 2025
- 2025Learning to Animate Images from A Few Videos to Portray Delicate Human Actions1 Mar 2025
- 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing24 Feb 2025
- 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Control2 Jan 2025
- 2024MAKIMA: Tuning-free Multi-Attribute Open-domain Video Editing via Mask-Guided Attention Modulation28 Dec 2024
5 more
- 2024PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Dataset17 Dec 2024
- 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices15 Oct 2024
- 2024EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models15 Sep 2024
- 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention29 Jul 2024
- 2024Dual-Personalizing Adapter for Federated Foundation Models28 Mar 2024
- 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
- 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editingcited by 50
- 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 39
- 2024EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Modelscited by 10
- 2026ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editingcited by 3
- 2024MAKIMA: Tuning-free Multi-Attribute Open-domain Video Editing via Mask-Guided Attention Modulationcited by 2
- 2025MOVi: Training-free Text-conditioned Multi-Object Video Generationcited by 0
- 2026ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing23 Jul 2026
- 2025MOVi: Training-free Text-conditioned Multi-Object Video Generation29 May 2025
- 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing24 Feb 2025
- 2024MAKIMA: Tuning-free Multi-Attribute Open-domain Video Editing via Mask-Guided Attention Modulation28 Dec 2024
- 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices15 Oct 2024
- 2024EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models15 Sep 2024
- 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention29 Jul 2024
- 2025VideoCoF: Unified Video Editing with Temporal Reasonercited by 5
- 2025VideoCoF: Unified Video Editing with Temporal Reasoner8 Dec 2025
- 2026Evolution of Video Generative Foundationscited by 6
- 2026Evolution of Video Generative Foundations7 Apr 2026
- 2024Dual-Personalizing Adapter for Federated Foundation Modelscited by 54
- 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Controlcited by 42
- 2025FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusioncited by 15
- 2025ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfercited by 12
- 2025Learning to Animate Images from A Few Videos to Portray Delicate Human Actionscited by 1
- 2024PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Datasetcited by 0
- 2025VIDMP3: Video Editing by Representing Motion with Pose and Position Priorscited by 0
- 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomycited by 0
- 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy28 Sep 2026
- 2025VIDMP3: Video Editing by Representing Motion with Pose and Position Priors14 Oct 2025
- 2025FreeLong++: Training-Free Long Video Generation via Multi-band SpectralFusion30 Jun 2025
- 2025ConMo: Controllable Motion Disentanglement and Recomposition for Zero-Shot Motion Transfer3 Apr 2025
- 2025Learning to Animate Images from A Few Videos to Portray Delicate Human Actions1 Mar 2025
- 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Control2 Jan 2025
- 2024PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Dataset17 Dec 2024
- 2024Dual-Personalizing Adapter for Federated Foundation Models28 Mar 2024
Similar, not cited
Close in wording, no citation either way
- 2024Investigating the Effectiveness of Cross-Attention to Unlock Zero-Shot Editing of Text-to-Video Diffusion Models
- 2024Object-Centric Diffusion for Efficient Video Editing
- 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models
- 2025Zero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence
- 2023InFusion: Inject and Attention Fusion for Multi Concept Zero-Shot Text-based Video Editing
- 2023RealCraft: Attention Control as A Tool for Zero-Shot Consistent Video Editing
- 2024Temporally Consistent Object Editing in Videos using Extended Attention
- 2025FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors