Wan-Weaver: Interleaved Multi-modal Generation via Decoupled Training
Not on the map. Why?
Abstract (arXiv)
Recent unified models have made unprecedented progress in both understanding and generation. However, while most of them accept multi-modal inputs, they typically produce only single-modality outputs. This challenge of producing interleaved content is mainly due to training data scarcity and the difficulty of modeling long-range cross-modal context. To address this issue, we decompose interleaved generation into textual planning and visual consistency modeling, and introduce a framework consisting of a planner and a visualizer. The planner produces dense textual descriptions for visual content, while the visualizer synthesizes images accordingly. Under this guidance, we construct large-scale textual-proxy interleaved data (where visual content is represented in text) to train the planner, and curate reference-guided image data to train the visualizer. These designs give rise to Wan-Weaver, which exhibits emergent interleaved generation ability with long-range textual coherence and visual consistency. Meanwhile, the integration of diverse understanding and generation data into planner training enables Wan-Weaver to achieve robust task reasoning and generation proficiency. To assess the model's capability in interleaved generation, we further construct a benchmark that spans a wide range of use cases across multiple dimensions. Extensive experiments demonstrate that, even without access to any real interleaved data, Wan-Weaver achieves superior performance over existing methods.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 73 of its 88 references that are in the map. First posted on arXiv 26 Mar 2026.
Builds on
73 of 88 references.
References · 73
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Visual Instruction Tuningcited by 7,792
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
38 more
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2022MaskGIT: Masked Generative Image Transformercited by 1,082
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2024Autoregressive Image Generation without Vector Quantizationcited by 644
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2024OmniGen: Unified Image Generationcited by 388
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
- 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
- 2023Generating Images with Multimodal Language Modelscited by 259
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
- 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
- 2025Transfer between Modalities with MetaQueriescited by 231
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
- 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
- 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
- 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 196
- 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184
- 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 168
- 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 151
- 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148
- 2025HunyuanImage 3.0 Technical Reportcited by 138
- 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
- 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
- 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generationcited by 112
50 most cited of 73 All 73 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026DuoGen: Towards General Purpose Interleaved Multimodal Generation
- 2026Wan-Image: Pushing the Boundaries of Generative Visual Intelligence
- 2026Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning
- 2025Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
- 2026Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement
- 2025WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
- 2024Instruct-Imagen: Image Generation with Multi-modal Instruction