# Depth-aware Test-Time Training for Zero-shot Video Object Segmentation

arXiv: https://arxiv.org/abs/2403.04258 · HTML: https://mlsanity.com/paper/2403.04258/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 63 references that are in the map.
- First posted on arXiv 7 Mar 2024.

## Builds on (most cited first, 1 of 1)

- [FateZero: Fusing Attentions for Zero-shot Text-based Video Editing](https://mlsanity.com/paper/2303.09535/) (2023)

## Built on by (most cited first, 27 of 27)

- [Underwater Image Restoration via Polymorphic Large Kernel CNNs](https://mlsanity.com/paper/2412.18459/) (2024)
- [High-Fidelity Document Stain Removal via A Large-Scale Real-World Dataset and A Memory-Augmented Transformer](https://mlsanity.com/paper/2410.22922/) (2024)
- [Test-Time Intensity Consistency Adaptation for Shadow Detection](https://mlsanity.com/paper/2410.07695/) (2024)
- [Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptation](https://mlsanity.com/paper/2505.20704/) (2025)
- [ForgeryTTT: Zero-Shot Image Manipulation Localization with Test-Time Training](https://mlsanity.com/paper/2410.04032/) (2024)
- [Implicit Multi-Spectral Transformer: An Lightweight and Effective Visible to Infrared Image Translation Model](https://mlsanity.com/paper/2404.07072/) (2024)
- [LensNet: An End-to-End Learning Framework for Empirical Point Spread Function Modeling and Lensless Imaging Reconstruction](https://mlsanity.com/paper/2505.01755/) (2025)
- [Training-Free Zero-Shot Temporal Action Detection with Vision-Language Models](https://mlsanity.com/paper/2501.13795/) (2025)
- [Order-aware Interactive Segmentation](https://mlsanity.com/paper/2410.12214/) (2024)
- [MAC-Lookup: Multi-Axis Conditional Lookup Model for Underwater Image Enhancement](https://mlsanity.com/paper/2507.02270/) (2025)
- [EEMS: Edge-Prompt Enhanced Medical Image Segmentation Based on Learnable Gating Mechanism](https://mlsanity.com/paper/2510.11287/) (2025)
- [Event Stream-based Visual Object Tracking: HDETrack V2 and A High-Definition Benchmark](https://mlsanity.com/paper/2502.05574/) (2025)
- [Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts](https://mlsanity.com/paper/2508.20488/) (2025)
- [GMOS: Grounding Moving Object Segmentation in 3D Space and Time](https://mlsanity.com/paper/2605.30352/) (2026)
- [Medical Visual Prompting (MVP): A Unified Framework for Versatile and High-Quality Medical Image Segmentation](https://mlsanity.com/paper/2404.01127/) (2024)
- [Adaptive Query Prompting for Multi-Domain Landmark Detection](https://mlsanity.com/paper/2404.01194/) (2024)
- [Enhanced Object Tracking by Self-Supervised Auxiliary Depth Estimation Learning](https://mlsanity.com/paper/2405.14195/) (2024)
- [TAGE: Trustworthy Attribute Group Editing for Stable Few-shot Image Generation](https://mlsanity.com/paper/2410.17855/) (2024)
- [Saliency-Motion Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2504.05904/) (2025)
- [DepthFlow: Exploiting Depth-Flow Structural Correlations for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2507.19790/) (2025)
- [Shallow Features Matter: Hierarchical Memory with Heterogeneous Interaction for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2507.22465/) (2025)
- [SFormer: SNR-guided Transformer for Underwater Image Enhancement from the Frequency Domain](https://mlsanity.com/paper/2508.18664/) (2025)
- [A Comprehensive Analysis of Churn Prediction in Telecommunications Using Machine Learning](https://mlsanity.com/paper/2509.22654/) (2025)
- [Forestpest-YOLO: A High-Performance Detection Framework for Small Forestry Pests](https://mlsanity.com/paper/2510.00547/) (2025)
- [DTEA: Dynamic Topology Weaving and Instability-Driven Entropic Attenuation for Medical Image Segmentation](https://mlsanity.com/paper/2510.11259/) (2025)
- [Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception](https://mlsanity.com/paper/2605.25651/) (2026)
- [VEDAL: Variational Error-Driven Asynchronous Learning for 3D Gaussian Splatting Pruning](https://mlsanity.com/paper/2606.02346/) (2026)

## Abstract (arXiv)

Zero-shot Video Object Segmentation (ZSVOS) aims at segmenting the primary moving object without any human annotations. Mainstream solutions mainly focus on learning a single model on large-scale video datasets, which struggle to generalize to unseen videos. In this work, we introduce a test-time training (TTT) strategy to address the problem. Our key insight is to enforce the model to predict consistent depth during the TTT process. In detail, we first train a single network to perform both segmentation and depth prediction tasks. This can be effectively learned with our specifically designed depth modulation layer. Then, for the TTT process, the model is updated by predicting consistent depth maps for the same frame under different data augmentations. In addition, we explore different TTT weight updating strategies. Our empirical results suggest that the momentum-based weight initialization and looping-based training scheme lead to more stable improvements. Experiments show that the proposed method achieves clear improvements on ZSVOS. Our proposed video TTT strategy provides significant superiority over state-of-the-art TTT methods. Our code is available at: https://nifangbaage.github.io/DATTT.