# StreamTTO: Efficient Online Test-Time Optimization for Video Depth Completion

arXiv: https://arxiv.org/abs/2603.01765 · HTML: https://mlsanity.com/paper/2603.01765/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 55 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 30 of 45)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details](https://mlsanity.com/paper/2507.02546/) (2025)
- [Video Depth Anything: Consistent Depth Estimation for Super-Long Videos](https://mlsanity.com/paper/2501.12375/) (2025)
- [Non-Local Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2007.10042/) (2020)
- [ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals](https://mlsanity.com/paper/1905.02082/) (2019)
- [UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler](https://mlsanity.com/paper/2502.20110/) (2025)
- [Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo](https://mlsanity.com/paper/2303.01943/) (2023)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [Streaming 4D Visual Geometry Transformer](https://mlsanity.com/paper/2507.11539/) (2025)
- [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018)
- [RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos](https://mlsanity.com/paper/2401.12592/) (2024)
- [Learning Guided Convolutional Network for Depth Completion](https://mlsanity.com/paper/1908.01238/) (2019)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [Real-time self-adaptive deep stereo](https://mlsanity.com/paper/1810.05424/) (2018)
- [Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation](https://mlsanity.com/paper/2412.14015/) (2024)
- [RoMa v2: Harder Better Faster Denser Feature Matching](https://mlsanity.com/paper/2511.15706/) (2025)

## Built on by (most cited first, 3 of 3)

- [Test-Time Adaptation for Height Completion via Self-Supervised ViT Features and Monocular Foundation Models](https://mlsanity.com/paper/2604.02009/) (2026)
- [Rethinking Electro-Optical Vision Foundation Models for Remote Sensing Retrieval: A Controlled Comparison with Generalist VFM](https://mlsanity.com/paper/2605.02283/) (2026)
- [T-VSS: Test-Time Visual Subspace Steering for Adversarial Robustness of Vision-Language Models](https://mlsanity.com/paper/2606.23132/) (2026)

## Abstract (arXiv)

Monocular depth foundation models generalize across diverse scenes, but recovering accurate metric depth consistent with a target sensor remains challenging under sensor variation and domain shift. Test-time optimization (TTO) guided by sparse depth addresses this limitation, yet independent per-frame adaptation incurs substantial computational cost. We propose StreamTTO, an online test-time optimization framework for video depth completion that reuses visual features and adaptation state. A frozen feature extractor with causal temporal attention incorporates past visual context into features cached for repeated decoder optimization. Causal Sliding-Window Optimization directly updates a shared depth decoder using cached features and sparse observations from recent frames, requiring only two passes over the window per incoming frame after initialization. Reusing recent observations and the adapted decoder substantially reduces optimization steps. We also introduce MS-Depth, comprising approximately 1.45M synchronized RGB--LiDAR frames from 40 hours of recordings. Its uninterrupted sequences capture transitions among indoor, underground, and outdoor environments under daytime and nighttime conditions, enabling evaluation of adaptation to changes in illumination, scene structure, and depth range. Experiments on KITTI, Bonn, NYUv2 Raw, TUM RGB-D, DDAD, and MS-Depth demonstrate competitive depth completion accuracy. On MS-Depth, StreamTTO achieves average processing speeds of approximately 15~FPS with VGGT and 25~FPS with MoGe-2 at an input resolution of 518 X 392.