mlsanityData to 14 Oct 2026

Continuous 3D Perception Model with Persistent State

arXiv 2501.12387 ↗First posted 21 Jan 2025cs.CVcited by 548

Not on the map. Why?

Cited by 34 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a unified framework capable of solving a broad range of 3D tasks. Our approach features a stateful recurrent model that continuously updates its state representation with each new observation. Given a stream of images, this evolving state can be used to generate metric-scale pointmaps (per-pixel 3D points) for each new input in an online fashion. These pointmaps reside within a common coordinate system, and can be accumulated into a coherent, dense scene reconstruction that updates as new images arrive. Our model, called CUT3R (Continuous Updating Transformer for 3D Reconstruction), captures rich priors of real-world scenes: not only can it predict accurate pointmaps from image observations, but it can also infer unseen regions of the scene by probing at virtual, unobserved views. Our method is simple yet highly flexible, naturally accepting varying lengths of images that may be either video streams or unordered photo collections, containing both static and dynamic content. We evaluate our method on various 3D/4D tasks and demonstrate competitive or state-of-the-art performance in each. Project Page: https://cut3r.github.io/

Key facts. Cited by 548 papers in the map (data through 14 Oct 2026). Builds on 75 of its 106 references that are in the map. First posted on arXiv 21 Jan 2025.

Builds on

75 of 106 references.

References · 75

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  9. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  11. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  12. 2021Vision Transformers for Dense Predictioncited by 1,803
38 more
  1. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  2. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  3. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  4. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  7. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  8. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  9. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  10. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  11. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  12. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  13. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  14. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  15. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  16. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  17. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
  18. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  19. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
  20. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  21. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  22. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
  23. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  24. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  25. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  26. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  27. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  28. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
  29. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  30. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  31. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
  32. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
  33. 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 326
  34. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 318
  35. 2021Neural RGB-D Surface Reconstructioncited by 313
  36. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
  37. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 299
  38. 20243D Reconstruction with Spatial Memorycited by 277

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 203; 2026: 345.

548 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  2. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  3. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 319
  4. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
  5. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 243
  6. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  7. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 166
  8. 2025Streaming 4D Visual Geometry Transformercited by 161
  9. 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 136
  10. 2025Video World Models with Long-term Spatial Memorycited by 136
  11. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
  12. 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequencescited by 115
38 more
  1. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 109
  2. 2025FastVGGT: Training-Free Acceleration of Visual Geometry Transformercited by 98
  3. 2025Aether: Geometric-Aware Unified World Modelingcited by 97
  4. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
  5. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 91
  6. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 89
  7. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 87
  8. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 87
  9. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 82
  10. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 77
  11. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
  12. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 69
  13. 2025Visual Imitation Enables Contextual Humanoid Controlcited by 68
  14. 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 67
  15. 2025Towards Understanding Camera Motions in Any Videocited by 64
  16. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 62
  17. 2026InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streamscited by 56
  18. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 54
  19. 2025Efficiently Reconstructing Dynamic Scenes One D4RT at a Timecited by 54
  20. 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Modelscited by 52
  21. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 52
  22. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 46
  23. 2025LONG3R: Long Sequence Streaming 3D Reconstructioncited by 45
  24. 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstructioncited by 44
  25. 2025G2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoningcited by 44
  26. 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worldscited by 44
  27. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  28. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  29. 2026LoGeR: Long-Context Geometric Reconstruction with Hybrid Memorycited by 39
  30. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 38
  31. 2025Human3R: Everyone Everywhere All at Oncecited by 37
  32. 2025YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splattingcited by 37
  33. 2026LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstructioncited by 37
  34. 2025UFM: A Simple Path towards Unified Dense Correspondence with Flowcited by 36
  35. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 35
  36. 2025WinT3R: Window-Based Streaming Reconstruction with Camera Token Poolcited by 35
  37. 2025Any4D: Unified Feed-Forward Metric 4D Reconstructioncited by 35
  38. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 34

50 most cited of 548 All 548 (JSON)