mlsanityData to 14 Oct 2026

Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling

arXiv 2608.29291 ↗First posted 29 Aug 2026cs.AIcited by 0

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Unified multimodal models jointly support understanding and generation, but incur substantial redundant computation across tokens, layers, and generation timesteps. Through token-importance probing, we identify an asymmetric core-expansion structure: understanding exhibits a stable importance component, while generation largely shares this component but requires progress-dependent corrections. We therefore propose CE-Router, which uses a task-shared core scorer and progress-conditioned generation expansions, optimized through generation decomposition and cross-task core alignment. At inference, CE-Router compacts token computation and supplies a learned routing signal to Unified Computation Scheduling, which coordinates layer skipping, FFN pruning, diffusion-head cache reuse, and denoising-step early exit. Experiments on two representative UMM architectures demonstrate consistent quality--efficiency improvements across both tasks, retaining 98.03% of dense understanding performance with a 1.93× end-to-end inference speedup.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 32 of its 42 references that are in the map. First posted on arXiv 29 Aug 2026.

Builds on

32 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026On the Limits of Token Reduction for Efficient Unified Vision Language Training31 May
  2. 2026Flash-Unified: A Training-Free and Task-Aware Acceleration Framework for Native Unified Models16 Mar
  3. 2026UniCompress: Token Compression for Unified Vision-Language Understanding and Generation11 Mar
  4. 2025UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths10 Feb
  5. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  6. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 26

  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  8. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  9. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  10. 2016A Diagram Is Worth A Dozen Imagescited by 804
  11. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  12. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
14 more
  1. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  2. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  3. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  4. 2023DeepCache: Accelerating Diffusion Models for Freecited by 359
  5. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  6. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  7. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  8. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 269
  9. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 258
  10. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  11. 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 185
  12. 2024DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perceptioncited by 73
  13. 2025ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusioncited by 12
  14. 2026Motion-Aware Caching for Efficient Autoregressive Video Generationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Understanding and Harnessing Sparsity in Unified Multimodal Models
  2. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligence
  3. 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scaling
  4. 2026UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding
  5. 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
  6. 2026From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning
  7. 2026Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference
  8. 2026CLEAR: Unlocking Generative Potential for Degraded Image Understanding in Unified Multimodal Models