mlsanityData to 14 Oct 2026

3D Reconstruction with Spatial Memory

arXiv 2408.16061 ↗First posted 28 Aug 2024cs.CVcited by 277

Not on the map. Why?

Abstract (arXiv)

We present Spann3R, a novel approach for dense 3D reconstruction from ordered or unordered image collections. Built on the DUSt3R paradigm, Spann3R uses a transformer-based architecture to directly regress pointmaps from images without any prior knowledge of the scene or camera parameters. Unlike DUSt3R, which predicts per image-pair pointmaps each expressed in its local coordinate frame, Spann3R can predict per-image pointmaps expressed in a global coordinate system, thus eliminating the need for optimization-based global alignment. The key idea of Spann3R is to manage an external spatial memory that learns to keep track of all previous relevant 3D information. Spann3R then queries this spatial memory to predict the 3D structure of the next frame in a global coordinate system. Taking advantage of DUSt3R's pre-trained weights, and further fine-tuning on a subset of datasets, Spann3R shows competitive performance and generalization ability on various unseen datasets and can process ordered image collections in real time. Project page: urlhttps://hengyiwang.github.io/projects/spanner

Key facts. Cited by 277 papers in the map (data through 14 Oct 2026). Builds on 63 of its 100 references that are in the map. First posted on arXiv 28 Aug 2024.

Builds on

63 of 100 references.

References · 63

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  6. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  7. 2021Vision Transformers for Dense Predictioncited by 1,803
  8. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  9. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  10. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  11. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
38 more
  1. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,076
  2. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  3. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  4. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  5. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  6. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  7. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
  8. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
  9. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  10. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  11. 2023LightGlue: Local Feature Matching at Light Speedcited by 590
  12. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
  13. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  14. 2023CoTracker: It is Better to Track Togethercited by 522
  15. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 490
  16. 2016Direct Sparse Odometrycited by 485
  17. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  18. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  19. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  20. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  21. 2023Gaussian Splatting SLAMcited by 415
  22. 2023SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAMcited by 388
  23. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  24. 2014Memory Networkscited by 363
  25. 2021Neural RGB-D Surface Reconstructioncited by 313
  26. 2016LIFT: Learned Invariant Feature Transformcited by 306
  27. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287
  28. 2016DSAC - Differentiable RANSAC for Camera Localizationcited by 266
  29. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
  30. 2023Visual Geometry Grounded Deep Structure From Motioncited by 227
  31. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
  32. 2024Global Structure-from-Motion Revisitedcited by 218
  33. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 200
  34. 2023Putting the Object Back into Video Object Segmentationcited by 188
  35. 2023Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAMcited by 184
  36. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  37. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 181
  38. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 166

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 118; 2026: 146.

277 papers.

Sort:

By area:

  1. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  2. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
  3. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 319
  4. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
  5. 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 205
  6. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 180
  7. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  8. 2025WorldMem: Long-term Consistent World Simulation with Memorycited by 168
  9. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 166
  10. 2025Streaming 4D Visual Geometry Transformercited by 161
  11. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 143
  12. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 141
38 more
  1. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 141
  2. 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 136
  3. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
  4. 2024SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videoscited by 102
  5. 2025Aether: Geometric-Aware Unified World Modelingcited by 97
  6. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 91
  7. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 87
  8. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 82
  9. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 82
  10. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  11. 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 72
  12. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 67
  13. 2026InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streamscited by 56
  14. 2025Light3R-SfM: Towards Feed-forward Structure-from-Motioncited by 47
  15. 2025DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoningcited by 46
  16. 2025LONG3R: Long Sequence Streaming 3D Reconstructioncited by 45
  17. 2025G2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoningcited by 44
  18. 2024FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstructioncited by 41
  19. 2026LoGeR: Long-Context Geometric Reconstruction with Hybrid Memorycited by 39
  20. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 38
  21. 2025Human3R: Everyone Everywhere All at Oncecited by 37
  22. 2026LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstructioncited by 37
  23. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 35
  24. 2025WinT3R: Window-Based Streaming Reconstruction with Camera Token Poolcited by 35
  25. 2026Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstructioncited by 31
  26. 2024Driv3R: Learning Dense 4D Reconstruction for Autonomous Drivingcited by 29
  27. 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
  28. 2024PreF3R: Pose-Free Feed-Forward 3D Gaussian Splatting from Variable-length Image Sequencecited by 28
  29. 2025VGGT4D: Mining Motion Cues in Visual Geometry Transformers for 4D Scene Reconstructioncited by 28
  30. 2026VGGT-SLAM 2.0: Real-time Dense Feed-forward Scene Reconstructioncited by 28
  31. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 27
  32. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
  33. 2025MVSAnywhere: Zero-Shot Multi-View Stereocited by 26
  34. 2026ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Trainingcited by 26
  35. 2025POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstructioncited by 25
  36. 20253DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Modelcited by 25
  37. 2025LongSplat: Robust Unposed 3D Gaussian Splatting for Casual Long Videoscited by 24
  38. 2025Block-Sparse Global Attention for Efficient Multi-View Geometry Transformerscited by 23

50 most cited of 277 All 277 (JSON)