mlsanityData to 7 Oct 2026

Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesis

arXiv 2312.13834 ↗First posted 20 Dec 2023cs.CVcited by 39

On the map: Diffusion models → Video generation

Abstract (arXiv)

In this paper, we introduce Fairy, a minimalist yet robust adaptation of image-editing diffusion models, enhancing them for video editing applications. Our approach centers on the concept of anchor-based cross-frame attention, a mechanism that implicitly propagates diffusion features across frames, ensuring superior temporal coherence and high-fidelity synthesis. Fairy not only addresses limitations of previous models, including memory and processing speed. It also improves temporal consistency through a unique data augmentation strategy. This strategy renders the model equivariant to affine transformations in both source and target images. Remarkably efficient, Fairy generates 120-frame 512x384 videos (4-second duration at 30 FPS) in just 14 seconds, outpacing prior works by at least 44x. A comprehensive user study, involving 1000 generated samples, confirms that our approach delivers superior quality, decisively outperforming established methods.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 26 of its 33 references that are in the map. First posted on arXiv 20 Dec 2023.

Builds on

26 of 33 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing18 Aug
  2. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation13 Jun
  3. 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models30 Mar
  4. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  5. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editing16 Mar
  6. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  7. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec

Other references · 19

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  7. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  8. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  9. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  10. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  11. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  12. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
7 more
  1. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  2. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  3. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 278
  4. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  5. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  6. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  7. 2023Shape-aware Text-driven Layered Video Editingcited by 54

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 17; 2025: 14; 2026: 7.

39 papers.

Sort:

By area:

  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  2. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
  3. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  4. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  5. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  6. 2025EditVerse: Unifying Image and Video Editing and Generation with In-Context Learningcited by 53
  7. 2024UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editingcited by 47
  8. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  9. 2024Video Editing via Factorized Diffusion Distillationcited by 40
  10. 2024Looking Backward: Streaming Video-to-Video Translation with Feature Bankscited by 38
  11. 2024Generative Video Propagationcited by 36
  12. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
27 more
  1. 2024AutoVFX: Physically Realistic Video Editing from Natural Language Instructionscited by 27
  2. 2025LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuningcited by 27
  3. 2024StableV2V: Stablizing Shape Consistency in Video-to-Video Editingcited by 25
  4. 2025Generative Physical AI in Vision: A Surveycited by 24
  5. 2024Mobile Video Diffusioncited by 17
  6. 2025SayAnything: Audio-Driven Lip Synchronization with Conditional Video Diffusioncited by 12
  7. 2025DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generationcited by 11
  8. 2024PRIME: Protect Your Videos From Malicious Editingcited by 9
  9. 2025RDPO: Real Data Preference Optimization for Physics Consistency Video Generationcited by 8
  10. 2025InstructVEdit: A Holistic Approach for Instructional Video Editingcited by 7
  11. 2025Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learningcited by 6
  12. 2025VideoCoF: Unified Video Editing with Temporal Reasonercited by 5
  13. 2025Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytellingcited by 4
  14. 2024Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunitiescited by 3
  15. 2024Negative Token Merging: Image-based Adversarial Feature Guidancecited by 3
  16. 2025Seeing Clearly, Forgetting Deeply: Revisiting Fine-Tuned Video Generators for Driving Simulationcited by 3
  17. 2024VIA: Unified Spatiotemporal Video Adaptation Framework for Global and Local Video Editingcited by 2
  18. 2024Beyond Generation: Unlocking Universal Editing via Self-Supervised Fine-Tuningcited by 2
  19. 2026NOVA: Sparse Control, Dense Synthesis for Pair-Free Video Editingcited by 2
  20. 2026RFDM: Residual Flow Diffusion Model for Efficient Causal Video Editingcited by 1
  21. 2026PropFly: Learning to Propagate via On-the-Fly Supervision from Pre-trained Video Diffusion Modelscited by 1
  22. 2026CityRAG: Stepping Into a City via Spatially-Grounded Video Generationcited by 1
  23. 2025Enhancing Video Inpainting with Aligned Frame Interval Guidancecited by 0
  24. 2025Driving scenario generation and evaluation using a structured layer representation and foundational modelscited by 0
  25. 2026Controllable Video Object Insertion via Multi-View Priorscited by 0
  26. 2026TeleMorpher: Toward Robust Simultaneous Motion-Location Editingcited by 0
  27. 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FastVideoEdit: Leveraging Consistency Models for Efficient Text-to-Video Editing
  2. 2025Efficient Temporal Consistency in Diffusion-Based Video Editing with Adaptor Modules: A Theoretical Framework
  3. 2024I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models
  4. 2026OSVE: One Step Video Editing with One Step Diffusion Models
  5. 2026Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editing
  6. 2026Efficient Video Diffusion Models: Advancements and Challenges
  7. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models
  8. 2023BIVDiff: A Training-Free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models