mlsanityData to 14 Oct 2026

EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning

arXiv 2509.20360 ↗First posted 24 Sep 2025cs.CVcited by 54

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advances in foundation models highlight a clear trend toward unification and scaling, showing emergent capabilities across diverse domains. While image generation and editing have rapidly transitioned from task-specific to unified frameworks, video generation and editing remain fragmented due to architectural limitations and data scarcity. In this work, we introduce EditVerse, a unified framework for image and video generation and editing within a single model. By representing all modalities, i.e., text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning, natural cross-modal knowledge transfer, and flexible handling of inputs and outputs with arbitrary resolutions and durations. To address the lack of video editing training data, we design a scalable data pipeline that curates 232K video editing samples and combines them with large-scale image and video datasets for joint training. Furthermore, we present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions. Extensive experiments and user studies demonstrate that EditVerse achieves state-of-the-art performance, surpassing existing open-source and commercial models, while exhibiting emergent editing and generation abilities across modalities.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 72 of its 83 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

72 of 83 references.

References · 72

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  10. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
38 more
  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  2. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  3. 2017Decoupled Weight Decay Regularizationcited by 1,831
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  5. 2024Depth Anything V2cited by 1,422
  6. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  7. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  8. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  10. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  11. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  12. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  13. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  14. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  15. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  16. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  17. 2025VACE: All-in-One Video Creation and Editingcited by 479
  18. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  19. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  20. 2024OmniGen: Unified Image Generationcited by 394
  21. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  22. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  23. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
  24. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
  25. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  26. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  27. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
  28. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
  29. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 204
  30. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 202
  31. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 193
  32. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 175
  33. 2023Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Codecited by 145
  34. 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervisioncited by 145
  35. 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 141
  36. 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Modelscited by 137
  37. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  38. 2023HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generationcited by 105

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 42.

54 papers.

Sort:

By area:

  1. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 87
  2. 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 79
  3. 2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 53
  4. 2026Rethinking Video Generation Model for the Embodied Worldcited by 33
  5. 2025PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generationcited by 23
  6. 2026Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editingcited by 19
  7. 2025VIVA: VLM-Guided Instruction-Based Video Editing with Reward Optimizationcited by 16
  8. 2026LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editingcited by 15
  9. 2025EasyV2V: A High-quality Instruction-based Video Editing Frameworkcited by 13
  10. 2026Lance: Unified Multimodal Modeling by Multi-Task Synergycited by 12
  11. 2026Bernini: Latent Semantic Planning for Video Diffusioncited by 12
  12. 2026Aurora: Unified Video Editing with a Tool-Using Agentcited by 9
38 more
  1. 2025EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editingcited by 8
  2. 2025VideoCoF: Unified Video Editing with Temporal Reasonercited by 8
  3. 2026SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editingcited by 8
  4. 2026SpongeBob: Sync-Aware Harmonious Audio-Visual Generative Editingcited by 8
  5. 2026RoboVIP: Multi-View Video Generation with Visual Identity Prompting Augments Robot Manipulationcited by 7
  6. 2026Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editingcited by 6
  7. 2025LoVoRA: Text-guided and Mask-free Video Object Removal and Addition with Learnable Object-aware Localizationcited by 5
  8. 2026FFP-300K: Scaling First-Frame Propagation for Generalizable Video Editingcited by 5
  9. 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editingcited by 5
  10. 2026LIVEditor-14B: Lightning Unified Video Editing via In-Context Sparse Attentioncited by 4
  11. 2026Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editingcited by 3
  12. 2026Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Controlcited by 3
  13. 2026Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformerscited by 3
  14. 2026Instruction-Based Video Editing by Repurposing an Image Editing Modelcited by 3
  15. 2025ViDiC: Video Difference Captioningcited by 2
  16. 2025CreativeVR: Diffusion-Prior-Guided Approach for Structure and Motion Restoration in Generative and Real Videoscited by 2
  17. 2026ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformercited by 2
  18. 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editingcited by 2
  19. 2025Refaccade: Editing Object with Given Reference Texturecited by 1
  20. 2026PLACID: Identity-Preserving Multi-Object Compositing via Video Diffusion with Synthetic Trajectoriescited by 1
  21. 2026FlexAM: Flexible Appearance-Motion Decomposition for Versatile Video Generation Controlcited by 1
  22. 2026PropFly: Learning to Propagate via On-the-Fly Supervision from Pre-trained Video Diffusion Modelscited by 1
  23. 2026Tri-Prompting: Video Diffusion with Unified Control over Scene, Subject, and Motioncited by 1
  24. 2026TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Spacecited by 1
  25. 2026TrajectoryMover: Generative Movement of Object Trajectories in Videoscited by 1
  26. 2026InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptationcited by 1
  27. 2026LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editingcited by 1
  28. 2026ATATA: One Algorithm to Align Them Allcited by 0
  29. 2026What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editingcited by 0
  30. 2026Occlusion-Aware Physics-Semantic Keyframe Selection for Robust Video Editingcited by 0
  31. 2026Chameleon: Style-Content Disentangled Framework for Cross-Domain Object Compositingcited by 0
  32. 2026Geometry-Instructed Video Editingcited by 0
  33. 2026FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Controlcited by 0
  34. 2026From Dense Prediction to Visual Editing: Structured Supervision for Unified Image and Video Creationcited by 0
  35. 2026VicEdit: Learning to Edit Videos from Visual In-Context Examplescited by 0
  36. 2026RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editingcited by 0
  37. 2026EditaLive! Unified Character Video Editing for Live Streamingcited by 0
  38. 2026VOR-Bench: A Human Perception-Driven Benchmark for Video Object Removalcited by 0

50 most cited of 54 All 54 (JSON)