VINO: A Unified Visual Generator with Interleaved OmniModal Context
Not on the map. Why?
Abstract (arXiv)
We present VINO, a unified visual generator that performs image and video generation and editing within a single framework. Instead of relying on task-specific models or independent modules for each modality, VINO uses a shared diffusion backbone that conditions on text, images and videos, enabling a broad range of visual creation and editing tasks under one model. Specifically, VINO couples a vision-language model (VLM) with a Multimodal Diffusion Transformer (MMDiT), where multimodal inputs are encoded as interleaved conditioning tokens, and then used to guide the diffusion process. This design supports multi-reference grounding, long-form instruction following, and coherent identity preservation across static and dynamic content, while avoiding modality-specific architectural components. To train such a unified system, we introduce a multi-stage training pipeline that progressively expands a video generation base model into a unified, multi-task generator capable of both image and video input and output. Across diverse generation and editing benchmarks, VINO demonstrates strong visual quality, faithful instruction following, improved reference and attribute preservation, and more controllable multi-identity edits. Our results highlight a practical path toward scalable unified visual generation, and the promise of interleaved, in-context computation as a foundation for general-purpose visual creation.
Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 79 of its 86 references that are in the map. First posted on arXiv 5 Jan 2026.
Builds on
79 of 86 references.
References · 79
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2023Visual Instruction Tuningcited by 7,869
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2024GPT-4o System Cardcited by 4,588
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
38 more
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
- 2025Qwen-Image Technical Reportcited by 1,044
- 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
- 2025VACE: All-in-One Video Creation and Editingcited by 479
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
- 2024OmniGen: Unified Image Generationcited by 394
- 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
- 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
- 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 249
50 most cited of 79 All 79 (JSON)
Built on by
Citing papers per year: 2026: 26.
26 papers.
Sort:
By area:
- 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoningcited by 27
- 2026Bernini: Latent Semantic Planning for Video Diffusioncited by 12
- 2026RefAlign: Representation Alignment for Reference-to-Video Generationcited by 10
- 2026SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editingcited by 8
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editingcited by 6
- 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusioncited by 6
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generationcited by 5
- 2026Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoEcited by 3
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationshipscited by 3
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generationcited by 2
- 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editingcited by 2
- 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editingcited by 2
14 more
- 2026How Long Can Unified Multimodal Models Generate Images Reliably? Taming Long-Horizon Interleaved Image Generation via Context Curationcited by 1
- 2026InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptationcited by 1
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancementcited by 1
- 2026VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learningcited by 0
- 2026CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognitioncited by 0
- 2026What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editingcited by 0
- 2026FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Controlcited by 0
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
- 2026From Dense Prediction to Visual Editing: Structured Supervision for Unified Image and Video Creationcited by 0
- 2026RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editingcited by 0
- 2026OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generationcited by 0
- 2026Reimagine Video Dynamicscited by 0
- 2026AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generationcited by 0
- 2026Transforming Image Editors into Video Editorscited by 0
- 2026Transforming Image Editors into Video Editors8 Oct 2026
- 2026AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generation7 Oct 2026
- 2026Reimagine Video Dynamics29 Sep 2026
- 2026OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation18 Sep 2026
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation10 Sep 2026
- 2026RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing26 Aug 2026
- 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing18 Aug 2026
- 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editing16 Aug 2026
- 2026From Dense Prediction to Visual Editing: Structured Supervision for Unified Image and Video Creation13 Aug 2026
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Sound6 Aug 2026
- 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion4 Aug 2026
- 2026FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control31 Jul 2026
14 more
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement20 Jul 2026
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships16 Jul 2026
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation6 Jun 2026
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing4 Jun 2026
- 2026Bernini: Latent Semantic Planning for Video Diffusion21 May 2026
- 2026What Semantics Survive the Connector? Diagnosing VLM-to-DiT Alignment in Video Editing20 May 2026
- 2026CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition19 May 2026
- 2026Mamoda2.5: Enhancing Unified Multimodal Model with DiT-MoE4 May 2026
- 2026VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning15 Apr 2026
- 2026InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation9 Apr 2026
- 2026RefAlign: Representation Alignment for Reference-to-Video Generation26 Mar 2026
- 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning25 Mar 2026
- 2026SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing19 Mar 2026
- 2026How Long Can Unified Multimodal Models Generate Images Reliably? Taming Long-Horizon Interleaved Image Generation via Context Curation8 Mar 2026
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationshipscited by 3
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generationcited by 2
- 2026InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptationcited by 1
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Soundcited by 0
- 2026Vorch-Omni: Multi-Task Orchestration of Sight and Sound6 Aug 2026
- 2026ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships16 Jul 2026
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation6 Jun 2026
- 2026InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation9 Apr 2026
- 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoningcited by 27
- 2026RefAlign: Representation Alignment for Reference-to-Video Generationcited by 10
- 2026SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editingcited by 8
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editingcited by 6
- 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusioncited by 6
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generationcited by 5
- 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editingcited by 2
- 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editingcited by 2
- 2026How Long Can Unified Multimodal Models Generate Images Reliably? Taming Long-Horizon Interleaved Image Generation via Context Curationcited by 1
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancementcited by 1
- 2026VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learningcited by 0
- 2026FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Controlcited by 0
12 most cited of 18 All 18 (JSON)
- 2026Transforming Image Editors into Video Editors8 Oct 2026
- 2026AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generation7 Oct 2026
- 2026Reimagine Video Dynamics29 Sep 2026
- 2026OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation18 Sep 2026
- 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation10 Sep 2026
- 2026RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing26 Aug 2026
- 2026CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing18 Aug 2026
- 2026EditStream: A Unified Autoregressive Framework for Interactive Video Generation and Editing16 Aug 2026
- 2026From Dense Prediction to Visual Editing: Structured Supervision for Unified Image and Video Creation13 Aug 2026
- 2026JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion4 Aug 2026
- 2026FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control31 Jul 2026
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement20 Jul 2026
12 newest of 18 All 18 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editing
- 2025UniVideo: Unified Understanding, Generation, and Editing for Videos
- 2025MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing
- 2026DuoGen: Towards General Purpose Interleaved Multimodal Generation
- 2025Query-Kontext: An Unified Multimodal Model for Image Generation and Editing
- 2026Beyond Text Conditioning: A Systematic Study of MLLM-DiT Fusion for Video Generation
- 2026HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer
- 2026Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing