mlsanityData to 14 Oct 2026

Lance: Unified Multimodal Modeling by Multi-Task Synergy

arXiv 2605.18678 ↗First posted 18 May 2026cs.CV cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

We present Lance, a lightweight native unified model supporting multimodal understanding, generation, and editing for both images and videos. Rather than relying on model capacity scaling or text-image-dominant designs, Lance explores a practical paradigm for unified multimodal modeling via collaborative multi-task training. It is grounded in two core principles: unified context modeling and decoupled capability pathways. Specifically, Lance is trained from scratch and employs a dual-stream mixture-of-experts architecture on shared interleaved multimodal sequences, enabling joint context learning while decoupling the pathways for understanding and generation. We further introduce modality-aware rotary positional encoding to mitigate interference among heterogeneous visual tokens and boost cross-task alignment. During training, Lance adopts a staged multi-task training paradigm with capability-oriented objectives and adaptive data scheduling to strengthen both semantic comprehension and visual generation performance. Experimental results demonstrate that Lance substantially outperforms existing open-source unified models in image and video generation, while retaining strong multimodal understanding capabilities. The homepage is available at https://lance-project.github.io.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 134 of its 146 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

134 of 146 references.

References · 134

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2025Qwen2.5-VL Technical Reportcited by 5,587
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  12. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2025Qwen3-VL Technical Reportcited by 2,793
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  12. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  13. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  14. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  15. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  16. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  17. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  18. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  19. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  20. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  21. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  22. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  23. 2025Qwen-Image Technical Reportcited by 1,044
  24. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  25. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  26. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  27. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  28. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  29. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  30. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  31. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  32. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  33. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  34. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  35. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  36. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  37. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  38. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661

50 most cited of 134 All 134 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 12.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation
  2. 2025ARMOR: Empowering Multimodal Understanding Model with Interleaved Multimodal Generation Capability
  3. 2026Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
  4. 2025OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models
  5. 2026DuoGen: Towards General Purpose Interleaved Multimodal Generation
  6. 2026Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steering
  7. 2025ToFu: Visual Tokens Reduction via Fusion for Multi-modal, Multi-patch, Multi-image Task
  8. 2026Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation