# Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformers

arXiv: https://arxiv.org/abs/2509.17650 · HTML: https://mlsanity.com/paper/2509.17650/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving.
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Placed in

- Efficient LLM inference and serving / [Token pruning and merging for ViTs](https://mlsanity.com/inference/token-pruning-and-merging-for-vits/)

## Builds on (most cited first, 18 of 18)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models](https://mlsanity.com/paper/2306.14048/) (2023)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time](https://mlsanity.com/paper/2305.17118/) (2023)
- [PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling](https://mlsanity.com/paper/2406.02069/) (2024)
- [Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass](https://mlsanity.com/paper/2501.13928/) (2025)
- [3D Reconstruction with Spatial Memory](https://mlsanity.com/paper/2408.16061/) (2024)
- [Streaming 4D Visual Geometry Transformer](https://mlsanity.com/paper/2507.11539/) (2025)
- [Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memory](https://mlsanity.com/paper/2507.02863/) (2025)
- [PuMer: Pruning and Merging Tokens for Efficient Vision Language Models](https://mlsanity.com/paper/2305.17530/) (2023)
- [D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models](https://mlsanity.com/paper/2406.13035/) (2024)

## Built on by (most cited first, 14 of 14)

- [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026)
- [AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend](https://mlsanity.com/paper/2511.20343/) (2025)
- [MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.15330/) (2026)
- [Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective](https://mlsanity.com/paper/2604.14025/) (2026)
- [OnlineHMR: Video-based Online World-Grounded Human Mesh Recovery](https://mlsanity.com/paper/2603.17355/) (2026)
- [On-the-fly Large-scale 3D Reconstruction from Multi-Camera Rigs](https://mlsanity.com/paper/2512.08498/) (2025)
- [FrameVGGT: Coherence-Preserving Memory for Bounded Streaming Geometry](https://mlsanity.com/paper/2603.07690/) (2026)
- [GHOST: Geometry-Hierarchical Online Streaming Token Eviction for Efficient 3D Reconstruction](https://mlsanity.com/paper/2605.15852/) (2026)
- [OVGGT: O(1) Constant-Cost Streaming Visual Geometry Transformer](https://mlsanity.com/paper/2603.05959/) (2026)
- [Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers](https://mlsanity.com/paper/2605.23892/) (2026)
- [VI3: Grounding Pretrained 3D Foundation Models with Inertial Cues](https://mlsanity.com/paper/2609.03824/) (2026)
- [Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction](https://mlsanity.com/paper/2609.04201/) (2026)
- [AMB3R-SLAM: Kilometer-scale SLAM with Hierarchical Backend](https://mlsanity.com/paper/2609.19518/) (2026)
- [RegVGGT: Sustainable Visual Geometry Grounding for Streaming via Regulated Memory](https://mlsanity.com/paper/2609.23286/) (2026)

## Abstract (arXiv)

Streaming visual transformers like StreamVGGT achieve strong 3D perception but suffer from unbounded growth of key value (KV) memory, which limits scalability. We propose a training-free, inference-time token eviction policy that bounds memory by discarding redundant tokens while keeping the most informative ones. Our method uses significantly less memory with little to no drop in accuracy: on 7-Scenes with long sequences it reduces peak memory from 18.63 GB to 9.39 GB while accuracy and completeness drop by only 0.003. Under strict memory budgets, eviction enables denser frame sampling, which improves reconstruction accuracy compared to the baseline. Experiments across video depth estimation (Sintel, KITTI), 3D reconstruction (7-Scenes, NRGBD), and camera pose estimation (Sintel, TUM-dynamics) show that our approach closely matches StreamVGGT at a fraction of the memory and makes long-horizon streaming inference more practical.