mlsanityData to 14 Oct 2026

VINO: A Unified Visual Generator with Interleaved OmniModal Context

arXiv 2601.02358 ↗First posted 5 Jan 2026cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

We present VINO, a unified visual generator that performs image and video generation and editing within a single framework. Instead of relying on task-specific models or independent modules for each modality, VINO uses a shared diffusion backbone that conditions on text, images and videos, enabling a broad range of visual creation and editing tasks under one model. Specifically, VINO couples a vision-language model (VLM) with a Multimodal Diffusion Transformer (MMDiT), where multimodal inputs are encoded as interleaved conditioning tokens, and then used to guide the diffusion process. This design supports multi-reference grounding, long-form instruction following, and coherent identity preservation across static and dynamic content, while avoiding modality-specific architectural components. To train such a unified system, we introduce a multi-stage training pipeline that progressively expands a video generation base model into a unified, multi-task generator capable of both image and video input and output. Across diverse generation and editing benchmarks, VINO demonstrates strong visual quality, faithful instruction following, improved reference and attribute preservation, and more controllable multi-identity edits. Our results highlight a practical path toward scalable unified visual generation, and the promise of interleaved, in-context computation as a foundation for general-purpose visual creation.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 79 of its 86 references that are in the map. First posted on arXiv 5 Jan 2026.

Builds on

79 of 86 references.

References · 79

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2022Classifier-Free Diffusion Guidancecited by 5,847
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2024GPT-4o System Cardcited by 4,588
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
38 more
  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  6. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  10. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  11. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  12. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  13. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  14. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  15. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  16. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  17. 2025Qwen-Image Technical Reportcited by 1,044
  18. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  19. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  20. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  21. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  22. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  23. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  24. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  25. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  26. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  27. 2025VACE: All-in-One Video Creation and Editingcited by 479
  28. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  29. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  30. 2024OmniGen: Unified Image Generationcited by 394
  31. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  32. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  33. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  34. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  35. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  36. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  37. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  38. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 249

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 26.

26 papers.

Sort:

By area:

  1. 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoningcited by 27
  2. 2026Bernini: Latent Semantic Planning for Video Diffusioncited by 12
  3. 2026RefAlign: Representation Alignment for Reference-to-Video Generationcited by 10
  4. 2026SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editingcited by 8
  5. 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editingcited by 6
  6. 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusioncited by 6
  7. 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generationcited by 5
  8. 2026Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoEcited by 3
  9. 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationshipscited by 3
  10. 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generationcited by 2
  11. 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editingcited by 2
  12. 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editingcited by 2
14 more
  1. 2026How Long Can Unified Multimodal Models Generate Images Reliably? Taming Long-Horizon Interleaved Image Generation via Context Curationcited by 1
  2. 2026InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptationcited by 1
  3. 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancementcited by 1
  4. 2026VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learningcited by 0
  5. 2026CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognitioncited by 0
  6. 2026What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editingcited by 0
  7. 2026FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Controlcited by 0
  8. 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
  9. 2026From Dense Prediction to Visual Editing: Structured Supervision for Unified Image and Video Creationcited by 0
  10. 2026RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editingcited by 0
  11. 2026OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generationcited by 0
  12. 2026Reimagine Video Dynamicscited by 0
  13. 2026AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generationcited by 0
  14. 2026Transforming Image Editors into Video Editorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editing
  2. 2025UniVideo: Unified Understanding, Generation, and Editing for Videos
  3. 2025MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing
  4. 2026DuoGen: Towards General Purpose Interleaved Multimodal Generation
  5. 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editing
  6. 2026Beyond Text Conditioning: A Systematic Study of MLLM-DiT Fusion for Video Generation
  7. 2026HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer
  8. 2026Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing