mlsanityData to 14 Oct 2026

Reasoning to Align: Implicit Reasoning in Diffusion Transformers for Video Editing

arXiv 2605.24674 ↗First posted 23 May 2026cs.CVcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Instruction-based video editing requires transforming a source video according to a natural-language instruction while preserving irrelevant content and remaining temporally coherent. We argue that existing Diffusion Transformer (DiT) editors struggle with this task for two structural reasons. First, conditioning signals are fed undifferentiated into all transformer blocks, forcing a single token stream to encode both global editing intent and fine-grained visual evidence. Second, the cross-attention patterns that govern the edit are supervised only indirectly through pixel-level reconstruction, leaving the model's internal reasoning process under-constrained. To address both limitations, we propose RVEDiT, an implicit Reasoning Video Editing DiT framework built around two complementary components. The first, Granularity-Routed Token Conditioning, introduces learnable editing tokens distilled from a multimodal LLM and routes them to shallow blocks, while reserving native visual and textual tokens for deeper blocks, thereby inducing a coarse-to-fine editing process inside the backbone. The second, Reference-Anchored Attention Alignment, employs a parameter-sharing reference branch during training and maximizes the mutual information between the attention features of the editing and reference branches, regularizing the model's internal reasoning without incurring any additional inference cost. Experiments on standard instruction-based video editing benchmarks show that RVEDiT consistently outperforms state-of-the-art baselines, with particularly strong gains on localized and compositional edits.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 55 of its 75 references that are in the map. First posted on arXiv 23 May 2026.

Builds on

55 of 75 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Meta-CoT: Enhancing Granularity and Generalization in Image Editing27 Apr
  2. 2025VEGGIE: Instructional Editing and Reasoning Video Concepts with Grounded Generation18 Mar

Other references · 53

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2022Flow Matching for Generative Modelingcited by 5,702
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  10. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  11. 2025Qwen3-VL Technical Reportcited by 2,793
  12. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
38 more
  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  3. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  4. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  5. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  6. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  7. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  8. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  9. 2025VACE: All-in-One Video Creation and Editingcited by 479
  10. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  11. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  12. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
  13. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  14. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 236
  15. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 173
  16. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 171
  17. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 154
  18. 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Modelscited by 142
  19. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 132
  20. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 129
  21. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 126
  22. 2023GenTron: Diffusion Transformers for Image and Video Generationcited by 82
  23. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 81
  24. 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 79
  25. 2023A-STAR: Test-time Attention Segregation and Retention for Text-to-image Synthesiscited by 78
  26. 2025Se~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialistscited by 61
  27. 2024I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Modelscited by 55
  28. 2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 53
  29. 2024Still-Moving: Customized Video Generation without Customized Video Datacited by 50
  30. 2023InstructVid2Vid: Controllable Video Editing with Natural Language Instructionscited by 44
  31. 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 40
  32. 2026Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidancecited by 36
  33. 2025In-Context Learning with Unpaired Clips for Instruction-based Video Editingcited by 35
  34. 2024Block Transformer: Global-to-Local Language Modeling for Fast Inferencecited by 32
  35. 2025Region-Constraint In-Context Generation for Instructional Video Editingcited by 23
  36. 2024EffiVED:Efficient Video Editing via Text-instruction Diffusion Modelscited by 21
  37. 2026Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editingcited by 19
  38. 2025VIVA: VLM-Guided Instruction-Based Video Editing with Reward Optimizationcited by 16

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Are Image-to-Video Models Good Zero-Shot Image Editors?
  2. 2026ThinkV2V: Unleashing the Reasoning Capability of MLLMs for Instruction-Guided Video Editing
  3. 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation
  4. 2025Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning
  5. 2025Text-Driven Reasoning Video Editing via Reinforcement Learning on Digital Twin Representations
  6. 2025Disentangling Instruction Influence in Diffusion Transformers for Parallel Multi-Instruction-Guided Image Editing
  7. 2026Aurora: Unified Video Editing with a Tool-Using Agent
  8. 2026Bernini: Latent Semantic Planning for Video Diffusion