mlsanityData to 7 Oct 2026

Streaming 4D Visual Geometry Transformer

arXiv 2507.11539 ↗First posted 15 Jul 2025cs.CV cs.AI cs.LGcited by 147

Not on the map. Why?

Abstract (arXiv)

Perceiving and reconstructing 3D geometry from videos is a fundamental yet challenging computer vision task. To facilitate interactive and low-latency applications, we propose a streaming visual geometry transformer that shares a similar philosophy with autoregressive large language models. We explore a simple and efficient design and employ a causal transformer architecture to process the input sequence in an online manner. We use temporal causal attention and cache the historical keys and values as implicit memory to enable efficient streaming long-term 3D reconstruction. This design can handle low-latency 3D reconstruction by incrementally integrating historical information while maintaining high-quality spatial consistency. For efficient training, we propose to distill knowledge from the dense bidirectional visual geometry grounded transformer (VGGT) to our causal model. For inference, our model supports the migration of optimized efficient attention operators (e.g., FlashAttention) from large language models. Extensive experiments on various 3D geometry perception benchmarks demonstrate that our model enhances inference speed in online scenarios while maintaining competitive performance, thereby facilitating scalable and interactive 3D vision systems. Code is available at: https://github.com/wzzheng/StreamVGGT.

Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). Builds on 51 of its 74 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

51 of 74 references.

References · 51

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2021Vision Transformers for Dense Predictioncited by 1,789
  6. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  7. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  8. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  9. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  10. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  11. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  12. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
38 more
  1. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  2. 2018Born Again Neural Networkscited by 539
  3. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  4. 2023CoTracker: It is Better to Track Togethercited by 516
  5. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  6. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  7. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  8. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 414
  9. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  10. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  11. 2020Virtual KITTI 2cited by 345
  12. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  13. 2021Neural RGB-D Surface Reconstructioncited by 312
  14. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  15. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  16. 20243D Reconstruction with Spatial Memorycited by 269
  17. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  18. 2022Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstructioncited by 211
  19. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 199
  20. 2021NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereocited by 191
  21. 2020Robust Consistent Video Depth Estimationcited by 188
  22. 2019ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residualscited by 180
  23. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 176
  24. 2024Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairscited by 171
  25. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 170
  26. 2023GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesiscited by 170
  27. 2017A Survey of Structure from Motioncited by 157
  28. 2023PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustmentcited by 146
  29. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  30. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 136
  31. 2015Modelling Uncertainty in Deep Learning for Camera Relocalizationcited by 133
  32. 2024RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videoscited by 113
  33. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  34. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 82
  35. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 81
  36. 2024EmbodiedOcc: Embodied 3D Occupancy Prediction for Vision-based Online Scene Understandingcited by 43
  37. 2024Robust Incremental Structure-from-Motion with Hybrid Featurescited by 31
  38. 2024Driv3R: Learning Dense 4D Reconstruction for Autonomous Drivingcited by 28

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 36; 2026: 111.

147 papers.

Sort:

By area:

  1. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 122
  2. 2025FastVGGT: Training-Free Acceleration of Visual Geometry Transformercited by 93
  3. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  4. 2026InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streamscited by 51
  5. 2025Human3R: Everyone Everywhere All at Oncecited by 35
  6. 2026LoGeR: Long-Context Geometric Reconstruction with Hybrid Memorycited by 33
  7. 2025WinT3R: Window-Based Streaming Reconstruction with Camera Token Poolcited by 32
  8. 2025VGGT4D: Mining Motion Cues in Visual Geometry Transformers for 4D Scene Reconstructioncited by 26
  9. 2026Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstructioncited by 25
  10. 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstructioncited by 24
  11. 2025SwiftVLA: Unlocking Spatiotemporal Dynamics for Lightweight VLA Models at Minimal Overheadcited by 23
  12. 2025DVGT: Driving Visual Geometry Transformercited by 20
38 more
  1. 2025RealX3D: A Physically-Degraded 3D Benchmark for Multi-view Visual Restoration and Reconstructioncited by 19
  2. 2025OmniVGGT: Omni-Modality Driven Visual Geometry Grounded Transformercited by 17
  3. 2026LongStream: Long-Sequence Streaming Autoregressive Visual Geometrycited by 17
  4. 2025EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmarkcited by 16
  5. 2025AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backendcited by 16
  6. 2025LiteVGGT: Boosting Vanilla VGGT via Geometry-aware Cached Token Mergingcited by 16
  7. 2025FlashVGGT: Efficient and Scalable Visual Geometry Transformers with Compressed Descriptor Attentioncited by 15
  8. 2026Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesiscited by 15
  9. 2026Vista4D: Video Reshooting with 4D Point Cloudscited by 15
  10. 2026VGG-T3: Offline Feed-Forward 3D Reconstruction at Scalecited by 14
  11. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  12. 2025Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformerscited by 13
  13. 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 12
  14. 2026TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstructioncited by 10
  15. 2026Track4World: Feedforward World-centric Dense 3D Tracking of All Pixelscited by 10
  16. 2026GlobalSplat: Efficient Feed-Forward 3D Gaussian Splatting via Global Scene Tokenscited by 10
  17. 2025Co-Me: Confidence-Guided Token Merging for Visual Geometric Transformerscited by 9
  18. 2026Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Predictioncited by 9
  19. 20254D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimationcited by 8
  20. 2026Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flowcited by 8
  21. 2026MoRe: Motion-aware Feed-forward 4D Reconstruction Transformercited by 8
  22. 2026STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstructioncited by 8
  23. 2025SLAM-Former: Putting SLAM into One Transformercited by 7
  24. 2025PAGE-4D: Disentangled pose and geometry estimation for vggt-4d perceptioncited by 7
  25. 2026UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Passcited by 7
  26. 2026XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compressioncited by 6
  27. 2026VGGT-Motion: Motion-Aware Calibration-Free Monocular SLAM for Long-Range Consistencycited by 6
  28. 2026Stream3D: Sequential Multi-View 3D Generation via Evidential Memorycited by 6
  29. 2026Global Structure-from-Motion Meets Feedforward Reconstructioncited by 6
  30. 2026Vision Pretraining for Dense Spatial Perceptioncited by 6
  31. 2025Captain Safari: A World Engine with Pose-Aligned 3D Memorycited by 5
  32. 20254DLangVGGT: 4D Language-Visual Geometry Grounded Transformercited by 5
  33. 2025On Geometric Understanding and Learned Priors in Feed-forward 3D Reconstruction Modelscited by 5
  34. 2025LASER: Layer-wise Scale Alignment for Training-Free Streaming 4D Reconstructioncited by 5
  35. 2025Reloc-VGGT: Visual Re-localization with Geometry Grounded Transformercited by 5
  36. 2025KV-Tracker: Real-Time Pose Tracking with Transformerscited by 5
  37. 2026ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstructioncited by 5
  38. 2026MeMix: Writing Less, Remembering More for Streaming 3D Reconstructioncited by 5

50 most cited of 147 All 147 (JSON)