mlsanityData to 7 Oct 2026

N"uwa: Mending the Spatial Integrity Torn by VLM Token Pruning

arXiv 2602.02951 ↗First posted 3 Feb 2026cs.CV cs.AI cs.CLcited by 13

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Vision token pruning has proven to be an effective acceleration technique for the efficient Vision Language Model (VLM). However, existing pruning methods demonstrate excellent performance preservation in visual question answering (VQA) and suffer substantial degradation on visual grounding (VG) tasks. Our analysis of the VLM's processing pipeline reveals that strategies utilizing global semantic similarity and attention scores lose the global spatial reference frame, which is derived from the interactions of tokens' positional information. Motivated by these findings, we propose N"uwa, a two-stage token pruning framework that enables efficient feature aggregation while maintaining spatial integrity. In the first stage, after the vision encoder, we apply three operations, namely separation, alignment, and aggregation, which are inspired by swarm intelligence algorithms to retain information-rich global spatial anchors. In the second stage, within the LLM, we perform text-guided pruning to retain task-relevant visual tokens. Extensive experiments demonstrate that N"uwa achieves SOTA performance on multiple VQA benchmarks (from 94% to 95%) and yields substantial improvements on visual grounding tasks (from 7% to 47%).

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 50 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

47 of 50 references.

Closest ancestors · 22

Same direction, newest first

  1. 2025LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compression30 Aug
  2. 2025HiPrune: Hierarchical Attention for Efficient Token Pruning in Vision-Language Models1 Aug
  3. 2025VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Models28 May
  4. 2025FlowCut: Rethinking Redundancy via Information Flow for Efficient Vision-Language Models26 May
  5. 2025Growing a Multi-head Twig via Distillation and Reinforcement Learning to Accelerate Large Vision-Language Models18 Mar
  6. 2025Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference17 Mar
  7. 2025Similarity-Aware Token Pruning: Your VLM but Faster14 Mar
  8. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models4 Mar
  9. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?17 Feb
  10. 2025Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Models9 Jan
  11. 2024FastVLM: Efficient Vision Encoding for Vision Language Models17 Dec
  12. 2024Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration17 Dec
10 more
  1. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  2. 2024Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification1 Dec
  3. 2024ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models30 Nov
  4. 2024Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model16 Nov
  5. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  6. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  7. 2024TokenPacker: Efficient Visual Projector for Multimodal LLM2 Jul
  8. 2024Matryoshka Multimodal Models27 May
  9. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  10. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 25

  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  8. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  11. 2019Towards VQA Models That Can Readcited by 1,696
  12. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
13 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  2. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  3. 2016Modeling Context in Referring Expressionscited by 1,054
  4. 2022Token Merging: Your ViT But Fastercited by 995
  5. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  6. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  7. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 195
  8. 2024Cross-modal Information Flow in Multimodal Large Language Modelscited by 65
  9. 2025EventVAD: Training-Free Event-Aware Video Anomaly Detectioncited by 32
  10. 2024Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputscited by 22
  11. 2025AdsQA: Towards Advertisement Video Understandingcited by 12
  12. 2024ProMerge: Prompt and Merge for Unsupervised Instance Segmentationcited by 10
  13. 2025Register and [CLS] tokens yield a decoupling of local and global features in large ViTscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026RTPrune: Reading-Twice Inspired Token Pruning for Efficient DeepSeek-OCR Inferencecited by 4
  2. 2026Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Modelscited by 2
  3. 2026DepthCache: Depth-Guided Training-Free Visual Token Merging for Vision-Language-Action Model Inferencecited by 1
  4. 2026ERASE: Eliminating Redundant Visual Tokens via Adaptive Two-Stage Token Pruningcited by 1
  5. 2026ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reductioncited by 1
  6. 2026PARCEL: Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient Vision-Language Understandingcited by 1
  7. 2026Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inferencecited by 1
  8. 2026Do VLMs Need Vision Transformers? Evaluating State Space Models as Vision Encoderscited by 0
  9. 2026CLASP: Class-Adaptive Layer Fusion and Dual-Stage Pruning for Multimodal Large Language Modelscited by 0
  10. 2026Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Modelscited by 0
  11. 2026Beyond Accuracy: Auditing Spatial Provenance in Visual Token Pruning for OCR-Critical MLLM Inferencecited by 0
  12. 2026Pay More Attention To Text In High-Resolution MLLMscited by 0
1 more
  1. 2026Still There, No Longer Seen: Exposing Compression-Induced Risk in Large Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm
  2. 2026IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruning
  3. 2026Beyond Token Importance: Preserving Spatial Scaffolds for Efficient Vision-Language-Action Inference
  4. 2025LVLM_CSP: Accelerating Large Vision Language Models via Clustering, Scattering, and Pruning for Reasoning Segmentation
  5. 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization
  6. 2026When Text Matters: Design Principles for Visual Token Pruning in Vision-Language Model
  7. 2026SwiftVLM: Efficient Vision-Language Model Inference via Cross-Layer Token Bypass
  8. 2026IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models