# TTT3R: 3D Reconstruction as Test-Time Training

arXiv: https://arxiv.org/abs/2509.26645 · HTML: https://mlsanity.com/paper/2509.26645/

## Key facts

- Cited by 132 papers in the map (data through 14 Oct 2026).
- Builds on 80 of its 107 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 30 of 80)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [Retentive Network: A Successor to Transformer for Large Language Models](https://mlsanity.com/paper/2307.08621/) (2023)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [Gated Linear Attention Transformers with Hardware-Efficient Training](https://mlsanity.com/paper/2312.06635/) (2023)
- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)

## Built on by (most cited first, 30 of 132)

- [InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams](https://mlsanity.com/paper/2601.02281/) (2026)
- [LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory](https://mlsanity.com/paper/2603.03269/) (2026)
- [Human3R: Everyone Everywhere All at Once](https://mlsanity.com/paper/2510.06219/) (2025)
- [LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction](https://mlsanity.com/paper/2604.14141/) (2026)
- [Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction](https://mlsanity.com/paper/2604.08542/) (2026)
- [VGGT-SLAM 2.0: Real-time Dense Feed-forward Scene Reconstruction](https://mlsanity.com/paper/2601.19887/) (2026)
- [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026)
- [ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training](https://mlsanity.com/paper/2603.04385/) (2026)
- [AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend](https://mlsanity.com/paper/2511.20343/) (2025)
- [RoboTTT: Context Scaling for Robot Policies](https://mlsanity.com/paper/2607.15275/) (2026)
- [ViT3: Unlocking Test-Time Training in Vision](https://mlsanity.com/paper/2512.01643/) (2025)
- [RealX3D: A Physically-Degraded 3D Benchmark for Multi-view Visual Restoration and Reconstruction](https://mlsanity.com/paper/2512.23437/) (2025)
- [VGG-T3: Offline Feed-Forward 3D Reconstruction at Scale](https://mlsanity.com/paper/2602.23361/) (2026)
- [FlashVGGT: Efficient and Scalable Visual Geometry Transformers with Compressed Descriptor Attention](https://mlsanity.com/paper/2512.01540/) (2025)
- [TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model](https://mlsanity.com/paper/2601.00051/) (2025)
- [Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesis](https://mlsanity.com/paper/2601.14253/) (2026)
- [AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories](https://mlsanity.com/paper/2602.14941/) (2026)
- [TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokens](https://mlsanity.com/paper/2604.15239/) (2026)
- [Test-Time Training with KV Binding Is Secretly Linear Attention](https://mlsanity.com/paper/2602.21204/) (2026)
- [Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels](https://mlsanity.com/paper/2603.02573/) (2026)
- [tttLRM: Test-Time Training for Long Context and Autoregressive 3D Reconstruction](https://mlsanity.com/paper/2602.20160/) (2026)
- [GlobalSplat: Efficient Feed-Forward 3D Gaussian Splatting via Global Scene Tokens](https://mlsanity.com/paper/2604.15284/) (2026)
- [TTSA3R: Training-Free Temporal-Spatial Adaptive Persistent State for Streaming 3D Reconstruction](https://mlsanity.com/paper/2601.22615/) (2026)
- [DROID-SLAM in the Wild](https://mlsanity.com/paper/2603.19076/) (2026)
- [VGGT-Motion: Motion-Aware Calibration-Free Monocular SLAM for Long-Range Consistency](https://mlsanity.com/paper/2602.05508/) (2026)
- [HTTM: Head-wise Temporal Token Merging for Faster VGGT](https://mlsanity.com/paper/2511.21317/) (2025)
- [VIGS-SLAM: Visual Inertial Gaussian Splatting SLAM](https://mlsanity.com/paper/2512.02293/) (2025)
- [HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction](https://mlsanity.com/paper/2605.23889/) (2026)
- [MUT3R: Motion-aware Updating Transformer for Dynamic 3D Reconstruction](https://mlsanity.com/paper/2512.03939/) (2025)
- [MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.15330/) (2026)

## Abstract (arXiv)

Modern Recurrent Neural Networks have become a competitive architecture for 3D reconstruction due to their linear-time complexity. However, their performance degrades significantly when applied beyond the training context length, revealing limited length generalization. In this work, we revisit the 3D reconstruction foundation models from a Test-Time Training perspective, framing their designs as an online learning problem. Building on this perspective, we leverage the alignment confidence between the memory state and incoming observations to derive a closed-form learning rate for memory updates, to balance between retaining historical information and adapting to new observations. This training-free intervention, termed TTT3R, substantially improves length generalization, achieving a 2× improvement in global pose estimation over baselines, while operating at 20 FPS with just 6 GB of GPU memory to process thousands of images. Code is available in https://rover-xingyu.github.io/TTT3R