# Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memory

arXiv: https://arxiv.org/abs/2507.02863 · HTML: https://mlsanity.com/paper/2507.02863/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 79 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 30 of 49)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass](https://mlsanity.com/paper/2501.13928/) (2025)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)

## Built on by (most cited first, 30 of 87)

- [Streaming 4D Visual Geometry Transformer](https://mlsanity.com/paper/2507.11539/) (2025)
- [TTT3R: 3D Reconstruction as Test-Time Training](https://mlsanity.com/paper/2509.26645/) (2025)
- [InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams](https://mlsanity.com/paper/2601.02281/) (2026)
- [Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft](https://mlsanity.com/paper/2510.03198/) (2025)
- [LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory](https://mlsanity.com/paper/2603.03269/) (2026)
- [Human3R: Everyone Everywhere All at Once](https://mlsanity.com/paper/2510.06219/) (2025)
- [WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool](https://mlsanity.com/paper/2509.05296/) (2025)
- [V-DPM: 4D Video Reconstruction with Dynamic Point Maps](https://mlsanity.com/paper/2601.09499/) (2026)
- [Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction](https://mlsanity.com/paper/2604.08542/) (2026)
- [Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey](https://mlsanity.com/paper/2507.14501/) (2025)
- [ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training](https://mlsanity.com/paper/2603.04385/) (2026)
- [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026)
- [DVGT: Driving Visual Geometry Transformer](https://mlsanity.com/paper/2512.16919/) (2025)
- [AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend](https://mlsanity.com/paper/2511.20343/) (2025)
- [VGG-T3: Offline Feed-Forward 3D Reconstruction at Scale](https://mlsanity.com/paper/2602.23361/) (2026)
- [Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesis](https://mlsanity.com/paper/2601.14253/) (2026)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformers](https://mlsanity.com/paper/2509.17650/) (2025)
- [DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale](https://mlsanity.com/paper/2604.00813/) (2026)
- [TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction](https://mlsanity.com/paper/2601.22615/) (2026)
- [Generalizing Visual Geometry Priors to Sparse Gaussian Occupancy Prediction](https://mlsanity.com/paper/2602.21552/) (2026)
- [4D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimation](https://mlsanity.com/paper/2511.18416/) (2025)
- [UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass](https://mlsanity.com/paper/2601.01222/) (2026)
- [STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.20284/) (2026)
- [HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction](https://mlsanity.com/paper/2605.23889/) (2026)
- [PAGE-4D: Disentangled pose and geometry estimation for vggt-4d perception](https://mlsanity.com/paper/2510.17568/) (2025)
- [MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.15330/) (2026)
- [MUT3R: Motion-aware Updating Transformer for Dynamic 3D Reconstruction](https://mlsanity.com/paper/2512.03939/) (2025)
- [XStreamVGGT: Extremely Memory-Efficient Streaming Vision Geometry Grounded Transformer with KV Cache Compression](https://mlsanity.com/paper/2601.01204/) (2026)
- [Stream3D: Sequential Multi-View 3D Generation via Evidential Memory](https://mlsanity.com/paper/2605.21472/) (2026)

## Abstract (arXiv)

Dense 3D scene reconstruction from an ordered sequence or unordered image collections is a critical step when bringing research in computer vision into practical scenarios. Following the paradigm introduced by DUSt3R, which unifies an image pair densely into a shared coordinate system, subsequent methods maintain an implicit memory to achieve dense 3D reconstruction from more images. However, such implicit memory is limited in capacity and may suffer from information loss of earlier frames. We propose Point3R, an online framework targeting dense streaming 3D reconstruction. To be specific, we maintain an explicit spatial pointer memory directly associated with the 3D structure of the current scene. Each pointer in this memory is assigned a specific 3D position and aggregates scene information nearby in the global coordinate system into a changing spatial feature. Information extracted from the latest frame interacts explicitly with this pointer memory, enabling dense integration of the current observation into the global coordinate system. We design a 3D hierarchical position embedding to promote this interaction and design a simple yet effective fusion mechanism to ensure that our pointer memory is uniform and efficient. Our method achieves competitive or state-of-the-art performance on various tasks with low training costs. Code: https://github.com/YkiWu/Point3R.