# FrameVGGT: Coherence-Preserving Memory for Bounded Streaming Geometry

arXiv: https://arxiv.org/abs/2603.07690 · HTML: https://mlsanity.com/paper/2603.07690/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving.
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 8 Mar 2026.

## Placed in

- Efficient LLM inference and serving / [Token pruning and merging for ViTs](https://mlsanity.com/inference/token-pruning-and-merging-for-vits/)

## Builds on (most cited first, 28 of 28)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [SnapKV: LLM Knows What You are Looking for Before Generation](https://mlsanity.com/paper/2404.14469/) (2024)
- [H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models](https://mlsanity.com/paper/2306.14048/) (2023)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time](https://mlsanity.com/paper/2305.17118/) (2023)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [MapAnything: Universal Feed-Forward Metric 3D Reconstruction](https://mlsanity.com/paper/2509.13414/) (2025)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [3D Reconstruction with Spatial Memory](https://mlsanity.com/paper/2408.16061/) (2024)
- [ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals](https://mlsanity.com/paper/1905.02082/) (2019)
- [MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion](https://mlsanity.com/paper/2409.19152/) (2024)
- [Streaming 4D Visual Geometry Transformer](https://mlsanity.com/paper/2507.11539/) (2025)
- [VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold](https://mlsanity.com/paper/2505.12549/) (2025)
- [TTT3R: 3D Reconstruction as Test-Time Training](https://mlsanity.com/paper/2509.26645/) (2025)
- [VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences](https://mlsanity.com/paper/2507.16443/) (2025)
- [SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videos](https://mlsanity.com/paper/2412.09401/) (2024)
- [Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memory](https://mlsanity.com/paper/2507.02863/) (2025)
- [InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams](https://mlsanity.com/paper/2601.02281/) (2026)
- [LONG3R: Long Sequence Streaming 3D Reconstruction](https://mlsanity.com/paper/2507.18255/) (2025)
- [WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool](https://mlsanity.com/paper/2509.05296/) (2025)
- [Robust Dense Mapping for Large-Scale Dynamic Environments](https://mlsanity.com/paper/1905.02781/) (2019)
- [SAIL-Recon: Large SfM by Augmenting Scene Regression with Localization](https://mlsanity.com/paper/2508.17972/) (2025)
- [Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformers](https://mlsanity.com/paper/2509.17650/) (2025)
- [XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression](https://mlsanity.com/paper/2601.01204/) (2026)
- [Sparse-then-Dense Alignment based 3D Map Reconstruction Method for Endoscopic Capsule Robots](https://mlsanity.com/paper/1708.09740/) (2017)

## Built on by (most cited first, 3 of 3)

- [FFVO: A Feedforward Pose Decoder for Long-Horizon Visual Odometry](https://mlsanity.com/paper/2609.13733/) (2026)
- [F2 SLAM: Turning Feed-Forward Geometry into Persistent Factors for SLAM](https://mlsanity.com/paper/2610.05207/) (2026)
- [Slot3R: Set-Associative Spatial Memory for Streaming 3D Reconstruction](https://mlsanity.com/paper/2610.12282/) (2026)

## Abstract (arXiv)

Streaming Visual Geometry Transformers such as StreamVGGT enable strong online 3D perception, but their KV-cache grows unbounded over long streams, limiting practical deployment. We study bounded-memory streaming geometry from the perspective of memory organization: unlike language modeling, where useful information can often be compressed at token level, geometry-driven inference relies on coherent and mutually compatible observations across views. Under fixed memory budgets, retaining history as isolated entries can progressively fragment the geometric context needed for stable long-horizon matching and fusion. We therefore propose FrameVGGT, a bounded-memory framework that maintains a fixed-capacity set of complementary memory units for streaming geometry. In our implementation, each unit is instantiated as a frame-wise KV segment summarized by a compact key-space prototype, together with a sparse anchor tier for persistent long-range references. Across long-sequence 3D reconstruction, video depth estimation, and camera pose estimation, FrameVGGT achieves favorable accuracy--memory trade-offs under bounded budgets while maintaining more stable geometry over long streams.