mlsanityData to 14 Oct 2026

QuickMerge++: Fast Token Merging with Autoregressive Prior

arXiv 2508.13204 ↗First posted 16 Aug 2025cs.AIcited by 1

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

As generative models scale to larger inputs across language, vision, and video domains, the cost of token-level computation has become a key bottleneck. While prior work suggests that only a subset of tokens significantly influence downstream predictions, most token selection methods are static, modality-specific, or incompatible with autoregressive generation. In this paper, we propose QuickMerge, a lightweight token merging framework designed for efficient next-token prediction. QuickMerge dynamically selects a reduced number of tokens based on attention norm magnitude, guided by an entropy-based budget estimator. To preserve autoregressive compatibility, we introduce a lightweight transformer prior trained over the merged token sequence. By combining semantic salience estimation, flexible token budgets, and AR alignment, QuickMerge enables accurate generation with fewer tokens. We evaluate QuickMerge across multi-modality domains, demonstrating consistent improvements in compute-accuracy tradeoffs. Specifically, QuickMerge reduces token counts sustantially while matching as well as exceeding the performance of learned tokenizers and fixed-patch baselines.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 37 references that are in the map. First posted on arXiv 16 Aug 2025.

Builds on

21 of 37 references.

References · 21

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2016Pointer Sentinel Mixture Modelscited by 3,096
  4. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  5. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  6. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  7. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  8. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  9. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 160
  10. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  11. 2023Token Fusion: Bridging the Gap between Token Pruning and Token Mergingcited by 125
  12. 2022Efficient Transformers with Dynamic Token Poolingcited by 76
9 more
  1. 2024LARP: Tokenizing Videos with a Learned Autoregressive Generative Priorcited by 46
  2. 2025FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximationcited by 24
  3. 2024MT2ST: Adaptive Multi-Task to Single-Task Learningcited by 16
  4. 2024LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inferencecited by 15
  5. 2025Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scalingcited by 10
  6. 2025Cloud-Based AI Systems: Leveraging Large Language Models for Intelligent Fault Detection and Autonomous Self-Healingcited by 10
  7. 2025Data Augmentation Through Random Style Replacementcited by 9
  8. 2025PiKV: KV Cache Management System for Mixture of Expertscited by 5
  9. 2024GraphSnapShot: Caching Local Structure for Fast Graph Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Cognitive Load Traces as Symbolic and Visual Accounts of Deep Model Cognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LapMerge: Differentiable Exact-Budget Token Merging in Linear Time
  2. 2026Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models
  3. 2026Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models
  4. 2026RUTA: Principled Visual Token Allocation via Rate-Utility Optimization
  5. 2026The Model Knows Which Tokens Matter: Automatic Token Selection via Noise Gating
  6. 2024Better & Faster Large Language Models via Multi-token Prediction
  7. 2026ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs
  8. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More