mlsanityData to 7 Oct 2026

Co-Me: Confidence-Guided Token Merging for Visual Geometric Transformers

arXiv 2511.14751 ↗First posted 18 Nov 2025cs.CV cs.ROcited by 9

On the map: Efficient LLM inference and serving → Token pruning and merging for ViTs

Abstract (arXiv)

We propose Confidence-Guided Token Merging (Co-Me), an acceleration mechanism for visual geometric transformers without retraining or finetuning the base model. Co-Me distilled a light-weight confidence predictor to rank tokens by uncertainty and selectively merge low-confidence ones, effectively reducing computation while maintaining spatial coverage. Compared to similarity-based merging or pruning, the confidence signal in Co-Me reliably indicates regions emphasized by the transformer, enabling substantial acceleration without degrading performance. Co-Me applies seamlessly to various multi-view and streaming visual geometric transformers, achieving speedups that scale with sequence length. When applied to VGGT and Pi3, Co-Me achieves up to 21.5x and 20.4x speedup, making visual geometric transformers practical for real-time 3D perception and reconstruction.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving. Builds on 30 of its 44 references that are in the map. First posted on arXiv 18 Nov 2025.

Builds on

30 of 44 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Token Fusion: Bridging the Gap between Token Pruning and Token Merging2 Dec
  2. 2023Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation2 Aug
  3. 2023Revisiting Token Pruning for Object Detection and Instance Segmentation12 Jun

Other references · 27

  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  2. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  3. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  4. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  5. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  6. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  7. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  8. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  9. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  10. 2017Sparsity Invariant CNNscited by 325
  11. 20243D Reconstruction with Spatial Memorycited by 269
  12. 2023Token Merging for Fast Stable Diffusioncited by 220
15 more
  1. 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
  2. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
  3. 2025Streaming 4D Visual Geometry Transformercited by 147
  4. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
  5. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  6. 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequencescited by 107
  7. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 81
  8. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  9. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  10. 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 69
  11. 2023X-Pruner: eXplainable Pruning for Vision Transformerscited by 66
  12. 2023SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformercited by 60
  13. 2023PuMer: Pruning and Merging Tokens for Efficient Vision Language Modelscited by 53
  14. 2019A Purely Algebraic Justification of the Kabsch-Umeyama Algorithmcited by 24
  15. 2024MAC-VO: Metrics-aware Covariance for Learning-based Stereo Visual Odometrycited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Foveated Diffusion: Efficient Spatially Adaptive Image and Video Generationcited by 2
  2. 2026PrePARE: Pre-AA Token Pruning for Frozen Multi-View Geometry Transformerscited by 0
  3. 2026Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformerscited by 0
  4. 2026SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformerscited by 0
  5. 2026Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstructioncited by 0
  6. 2026FFVO: A Feedforward Pose Decoder for Long-Horizon Visual Odometrycited by 0
  7. 2026ReSS: Residual-Restoring Sparse Attention for 3D Vision Transformerscited by 0
  8. 2026OTT3R: Multi-View 3D Reconstruction and Fast Dataset Generation at 1% Computecited by 0
  9. 2026Does the VGGT Family Need All Its Layers?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025FastVGGT: Training-Free Acceleration of Visual Geometry Transformer
  2. 2026MergeOver: Post-Training Token Merging for Recursive Vision Transformers
  3. 2025LiteVGGT: Boosting Vanilla VGGT via Geometry-aware Cached Token Merging
  4. 2023PPT: Token Pruning and Pooling for Efficient Vision Transformers
  5. 2026VGGT-Prime: Compute-Adaptive Mixture-of-Heads for Efficient Visual Geometry Transformers
  6. 2025HTTM: Head-wise Temporal Token Merging for Faster VGGT
  7. 2023Training-Free Acceleration of ViTs with Delayed Spatial Merging
  8. 2026Fusion: A Framework for Unified Sequential Token AdaptatIon in VisiOn TraNsformers