mlsanityData to 7 Oct 2026

Wan-Weaver: Interleaved Multi-modal Generation via Decoupled Training

arXiv 2603.25706 ↗First posted 26 Mar 2026cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Recent unified models have made unprecedented progress in both understanding and generation. However, while most of them accept multi-modal inputs, they typically produce only single-modality outputs. This challenge of producing interleaved content is mainly due to training data scarcity and the difficulty of modeling long-range cross-modal context. To address this issue, we decompose interleaved generation into textual planning and visual consistency modeling, and introduce a framework consisting of a planner and a visualizer. The planner produces dense textual descriptions for visual content, while the visualizer synthesizes images accordingly. Under this guidance, we construct large-scale textual-proxy interleaved data (where visual content is represented in text) to train the planner, and curate reference-guided image data to train the visualizer. These designs give rise to Wan-Weaver, which exhibits emergent interleaved generation ability with long-range textual coherence and visual consistency. Meanwhile, the integration of diverse understanding and generation data into planner training enables Wan-Weaver to achieve robust task reasoning and generation proficiency. To assess the model's capability in interleaved generation, we further construct a benchmark that spans a wide range of use cases across multiple dimensions. Extensive experiments demonstrate that, even without access to any real interleaved data, Wan-Weaver achieves superior performance over existing methods.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 73 of its 88 references that are in the map. First posted on arXiv 26 Mar 2026.

Builds on

73 of 88 references.

References · 73

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  10. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
38 more
  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  3. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  4. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  5. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  7. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  8. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  9. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  10. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  11. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  12. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  13. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  14. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  15. 2024OmniGen: Unified Image Generationcited by 388
  16. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  17. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  18. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  19. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  20. 2023Generating Images with Multimodal Language Modelscited by 259
  21. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  22. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  23. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  24. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  25. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
  26. 2025Transfer between Modalities with MetaQueriescited by 231
  27. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  28. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  29. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  30. 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 196
  31. 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184
  32. 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 168
  33. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 151
  34. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148
  35. 2025HunyuanImage 3.0 Technical Reportcited by 138
  36. 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
  37. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  38. 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generationcited by 112

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026DuoGen: Towards General Purpose Interleaved Multimodal Generation
  2. 2026Wan-Image: Pushing the Boundaries of Generative Visual Intelligence
  3. 2026Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning
  4. 2025Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
  5. 2026Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement
  6. 2025WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation
  7. 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
  8. 2024Instruct-Imagen: Image Generation with Multi-modal Instruction