mlsanityData to 7 Oct 2026

STAR: STacked AutoRegressive Scheme for Unified Multimodal Learning

arXiv 2512.13752 ↗First posted 15 Dec 2025cs.CV cs.AIcited by 6

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal large language models (MLLMs) play a pivotal role in advancing the quest for general artificial intelligence. However, achieving unified target for multimodal understanding and generation remains challenging due to optimization conflicts and performance trade-offs. To effectively enhance generative performance while preserving existing comprehension capabilities, we introduce STAR: a STacked AutoRegressive scheme for task-progressive unified multimodal learning. This approach decomposes multimodal learning into multiple stages: understanding, generation, and editing. By freezing the parameters of the fundamental autoregressive (AR) model and progressively stacking isomorphic AR modules, it avoids cross-task interference while expanding the model's capabilities. Concurrently, we introduce a high-capacity VQ to enhance the granularity of image representations and employ an implicit reasoning mechanism to improve generation quality under complex conditions. Experiments demonstrate that STAR achieves state-of-the-art performance on GenEval (0.91), DPG-Bench (87.44), and ImgEdit (4.34), validating its efficacy for unified multimodal learning.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 61 references that are in the map. First posted on arXiv 15 Dec 2025.

Builds on

55 of 61 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023DreamLLM: Synergistic Multimodal Comprehension and Creation20 Sep
  2. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension30 Jul
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun

Other references · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  12. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
38 more
  1. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  3. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  4. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  5. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  6. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  7. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  8. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  9. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  10. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  11. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
  12. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  13. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  14. 2024OmniGen: Unified Image Generationcited by 388
  15. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  16. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  17. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
  18. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  19. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  20. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  21. 2025Transfer between Modalities with MetaQueriescited by 231
  22. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  23. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  24. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  25. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  26. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  27. 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 191
  28. 2023Emu: Generative Pretraining in Multimodalitycited by 156
  29. 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervisioncited by 145
  30. 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
  31. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  32. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
  33. 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
  34. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 93
  35. 2025Ovis-U1 Technical Reportcited by 65
  36. 2023Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulationcited by 61
  37. 2025ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinementcited by 53
  38. 2025An Empirical Study of GPT-4o Image Generation Capabilitiescited by 25

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editingcited by 14
  2. 2026BlackMirror: Black-Box Backdoor Detection for Text-to-Image Models via Instruction-Response Deviationcited by 1
  3. 2026KALAVAI: Predicting When Independent Specialist Fusion Works -- A Quantitative Model for Post-Hoc Cooperative LLM Trainingcited by 0
  4. 2026UniVR: Thinking in Visual Space for Unified Visual Reasoningcited by 0
  5. 2026Test-Time Curriculum for Open-Set AIGC Detectioncited by 0
  6. 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AR-Omni: A Unified Autoregressive Model for Any-to-Any Generation
  2. 2026UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm
  3. 2025VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model
  4. 2025Unified Multimodal Discrete Diffusion
  5. 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
  6. 2024The Revolution of Multimodal Large Language Models: A Survey
  7. 2025ARMOR: Empowering Multimodal Understanding Model with Interleaved Multimodal Generation Capability
  8. 2025Growing Visual Generative Capacity for Pre-Trained MLLMs