# FastVGGT: Training-Free Acceleration of Visual Geometry Transformer

arXiv: https://arxiv.org/abs/2509.02560 · HTML: https://mlsanity.com/paper/2509.02560/

## Key facts

- Cited by 98 papers in the map (data through 14 Oct 2026).
- Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving.
- Builds on 29 of its 33 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Placed in

- Efficient LLM inference and serving / [Token pruning and merging for ViTs](https://mlsanity.com/inference/token-pruning-and-merging-for-vits/)

## Builds on (most cited first, 29 of 29)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass](https://mlsanity.com/paper/2501.13928/) (2025)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [Token Merging for Fast Stable Diffusion](https://mlsanity.com/paper/2303.17604/) (2023)
- [FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views](https://mlsanity.com/paper/2502.12138/) (2025)
- [Streaming 4D Visual Geometry Transformer](https://mlsanity.com/paper/2507.11539/) (2025)
- [MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds](https://mlsanity.com/paper/2412.06974/) (2024)
- [VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences](https://mlsanity.com/paper/2507.16443/) (2025)
- [Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer](https://mlsanity.com/paper/2204.08680/) (2022)
- [Learning to Merge Tokens in Vision Transformers](https://mlsanity.com/paper/2202.12015/) (2022)
- [Which Tokens to Use? Investigating Token Reduction in Vision Transformers](https://mlsanity.com/paper/2308.04657/) (2023)
- [PuMer: Pruning and Merging Tokens for Efficient Vision Language Models](https://mlsanity.com/paper/2305.17530/) (2023)
- [vid-TLDR: Training Free Token merging for Light-weight Video Transformer](https://mlsanity.com/paper/2403.13347/) (2024)
- [TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval](https://mlsanity.com/paper/2409.01156/) (2024)
- [Accelerating Transformers with Spectrum-Preserving Token Merging](https://mlsanity.com/paper/2405.16148/) (2024)
- [Video Token Merging for Long-form Video Understanding](https://mlsanity.com/paper/2410.23782/) (2024)
- [Evolving High-Quality Rendering and Reconstruction in a Unified Framework with Contribution-Adaptive Regularization](https://mlsanity.com/paper/2503.00881/) (2025)
- [DeOcc-1-to-3: 3D De-Occlusion from a Single Image via Self-Supervised Multi-View Diffusion](https://mlsanity.com/paper/2506.21544/) (2025)

## Built on by (most cited first, 30 of 98)

- [WorldMirror: Universal 3D World Reconstruction with Any-Prior Prompting](https://mlsanity.com/paper/2510.10726/) (2025)
- [InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams](https://mlsanity.com/paper/2601.02281/) (2026)
- [LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory](https://mlsanity.com/paper/2603.03269/) (2026)
- [LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction](https://mlsanity.com/paper/2604.14141/) (2026)
- [The Better You Learn, The Smarter You Prune: Towards Efficient Vision-language-action Models via Differentiable Token Pruning](https://mlsanity.com/paper/2509.12594/) (2025)
- [Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction](https://mlsanity.com/paper/2601.04090/) (2026)
- [Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction](https://mlsanity.com/paper/2604.08542/) (2026)
- [VGGT4D: Mining Motion Cues in Visual Geometry Transformers for 4D Scene Reconstruction](https://mlsanity.com/paper/2511.19971/) (2025)
- [Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching](https://mlsanity.com/paper/2512.11130/) (2025)
- [ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training](https://mlsanity.com/paper/2603.04385/) (2026)
- [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026)
- [LongStream: Long-Sequence Streaming Autoregressive Visual Geometry](https://mlsanity.com/paper/2602.13172/) (2026)
- [VGG-T3: Offline Feed-Forward 3D Reconstruction at Scale](https://mlsanity.com/paper/2602.23361/) (2026)
- [Quantized Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2509.21302/) (2025)
- [FlashVGGT: Efficient and Scalable Visual Geometry Transformers with Compressed Descriptor Attention](https://mlsanity.com/paper/2512.01540/) (2025)
- [LiteVGGT: Boosting Vanilla VGGT via Geometry-aware Cached Token Merging](https://mlsanity.com/paper/2512.04939/) (2025)
- [AVGGT: Rethinking Global Attention for Accelerating VGGT](https://mlsanity.com/paper/2512.02541/) (2025)
- [TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction](https://mlsanity.com/paper/2601.22615/) (2026)
- [Speed3R: Sparse Feed-forward 3D Reconstruction Models](https://mlsanity.com/paper/2603.08055/) (2026)
- [HTTM: Head-wise Temporal Token Merging for Faster VGGT](https://mlsanity.com/paper/2511.21317/) (2025)
- [STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.20284/) (2026)
- [HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction](https://mlsanity.com/paper/2605.23889/) (2026)
- [MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentation](https://mlsanity.com/paper/2601.06874/) (2026)
- [MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.15330/) (2026)
- [Global Structure-from-Motion Meets Feedforward Reconstruction](https://mlsanity.com/paper/2605.26103/) (2026)
- [Emergent Extreme-View Geometry in 3D Foundation Models](https://mlsanity.com/paper/2511.22686/) (2025)
- [Analyzing the Mechanism of Attention Collapse in VGGT from a Dynamics Perspective](https://mlsanity.com/paper/2512.21691/) (2025)
- [ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstruction](https://mlsanity.com/paper/2603.07552/) (2026)
- [VGGT-360: Geometry-Consistent Zero-Shot Panoramic Depth Estimation](https://mlsanity.com/paper/2603.18943/) (2026)
- [HeSS: Head Sensitivity Score for Sparsity Redistribution in VGGT](https://mlsanity.com/paper/2603.25336/) (2026)

## Abstract (arXiv)

Foundation models for 3D vision have recently demonstrated remarkable capabilities in 3D perception. However, scaling these models to long-sequence image inputs remains a significant challenge due to inference-time inefficiency. In this work, we present a detailed analysis of VGGT, a state-of-the-art feed-forward visual geometry model and identify its primary bottleneck. Visualization further reveals a token collapse phenomenon in the attention maps. Motivated by these findings, we explore the potential of token merging in the feed-forward visual geometry model. Owing to the unique architectural and task-specific properties of 3D models, directly applying existing merging techniques proves challenging. To this end, we propose FastVGGT, which, for the first time, leverages token merging in the 3D domain through a training-free mechanism for accelerating VGGT. we devise a unique token partitioning strategy tailored to 3D architectures and tasks, effectively eliminating redundant computation while preserving VGGT's powerful reconstruction capacity. Extensive experiments on multiple 3D geometry benchmarks validate the effectiveness of our approach. Notably, with 1000 input images, FastVGGT achieves a 4x speedup over VGGT while mitigating error accumulation in long-sequence scenarios. These findings underscore the potential of token merging as a principled solution for scalable 3D vision systems. Code is available at: https://mystorm16.github.io/fastvggt/.