mlsanityData to 7 Oct 2026

ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training

arXiv 2603.04385 ↗First posted 4 Mar 2026cs.CV cs.AI cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Feed-forward transformer models have driven rapid progress in 3D vision, but state-of-the-art methods such as VGGT and π3 have a computational cost that scales quadratically with the number of input images, making them inefficient when applied to large image collections. Sequential-reconstruction approaches reduce this cost but sacrifice reconstruction quality. We introduce ZipMap, a stateful feed-forward model that achieves linear-time, bidirectional 3D reconstruction while matching or surpassing the accuracy of quadratic-time methods. ZipMap employs test-time training layers to zip an entire image collection into a compact hidden scene state in a single forward pass, enabling reconstruction of over 700 frames in under 10 seconds on a single H100 GPU, more than 20× faster than state-of-the-art methods such as VGGT. Moreover, we demonstrate the benefits of having a stateful representation in real-time scene-state querying and its extension to sequential streaming reconstruction.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 65 of its 86 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

65 of 86 references.

References · 65

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  8. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  9. 2021Vision Transformers for Dense Predictioncited by 1,789
  10. 2020GLU Variants Improve Transformercited by 1,724
  11. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  12. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
38 more
  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  2. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  3. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  4. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  5. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  6. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  7. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  8. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  9. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  10. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  11. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  12. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  13. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  14. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  15. 2022Kubric: A scalable dataset generatorcited by 373
  16. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
  17. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  18. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  19. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  20. 2021Neural RGB-D Surface Reconstructioncited by 312
  21. 2024Titans: Learning to Memorize at Test Timecited by 303
  22. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  23. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  24. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  25. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  26. 20243D Reconstruction with Spatial Memorycited by 269
  27. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  28. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 241
  29. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
  30. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 229
  31. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  32. 2024Global Structure-from-Motion Revisitedcited by 215
  33. 2023MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyondcited by 185
  34. 2019ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residualscited by 180
  35. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 178
  36. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 176
  37. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 170
  38. 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 23.

23 papers.

Sort:

By area:

  1. 2026VGGT-Omegacited by 20
  2. 2026Mem3R: Streaming 3D Reconstruction with Hybrid Memory via Test-Time Trainingcited by 5
  3. 2026Long-tail Internet photo reconstructioncited by 4
  4. 2026Mamba-VGGT: Persistent Long-Sequence Video Geometry Grounded Transformer via External Sliding Window Mamba Memorycited by 3
  5. 2026X-Lens: Real-Time Metric Depth Estimation with Heterogeneous Camerascited by 3
  6. 2026R3: 3D Reconstruction via Relative Regressioncited by 2
  7. 2026G3T Up! Gravity Aligned Coordinate Frames Simplify Pointmap Processingcited by 2
  8. 2026Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstructioncited by 1
  9. 2026TurboVGGT: Fast Visual Geometry Reconstruction with Adaptive Alternating Attentioncited by 1
  10. 2026Rethinking the State Update Gate for Long-Sequence Recurrent 3D Reconstructioncited by 1
  11. 2026Compressing History into Memory: Distilling Transformers into Recurrent Transformerscited by 1
  12. 2026GeoFF3D: Coordinate-Anchored Feed-Forward Reconstruction for Large-Scale UAV Mappingcited by 1
11 more
  1. 2026Point4D: Long-range 4D Motion Reconstructioncited by 1
  2. 2026Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformerscited by 0
  3. 2026SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformerscited by 0
  4. 2026Graph Sparse Sampling: Breaking the Curse of the Horizon in Continuous MDP Planningcited by 0
  5. 2026Online Neural Space Time Memory for Dynamic Novel View Synthesiscited by 0
  6. 2026GeoWeaver: Accurate Long-Sequence 3D Reconstruction via Hierarchical Geometric Assemblycited by 0
  7. 2026VI3: Grounding Pretrained 3D Foundation Models with Inertial Cuescited by 0
  8. 2026Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Renderingcited by 0
  9. 2026TTTIR: Unlocking Instance-Specific State Evolution via Test-Time Training for Image Restorationcited by 0
  10. 2026CLoSeR: Closing the Loop for Long-Context Streaming Reconstructioncited by 0
  11. 2026Slot3R: Set-Associative Spatial Memory for Streaming 3D Reconstructioncited by 0