mlsanityData to 14 Oct 2026

Beyond Appearance: Geometric Cues for Robust Video Instance Segmentation

arXiv 2507.05948 ↗First posted 8 Jul 2025cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Video Instance Segmentation (VIS) fundamentally struggles with pervasive challenges including object occlusions, motion blur, and appearance variations during temporal association. To overcome these limitations, this work introduces geometric awareness to enhance VIS robustness by strategically leveraging monocular depth estimation. We systematically investigate three distinct integration paradigms. Expanding Depth Channel (EDC) method concatenates the depth map as input channel to segmentation networks; Sharing ViT (SV) designs a uniform ViT backbone, shared between depth estimation and segmentation branches; Depth Supervision (DS) makes use of depth prediction as an auxiliary training guide for feature learning. Though DS exhibits limited effectiveness, benchmark evaluations demonstrate that EDC and SV significantly enhance the robustness of VIS. When with Swin-L backbone, our EDC method gets 56.2 AP, which sets a new state-of-the-art result on OVIS benchmark. This work conclusively establishes depth cues as critical enablers for robust video understanding.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 54 of its 63 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

54 of 63 references.

References · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  10. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  11. 2021Vision Transformers for Dense Predictioncited by 1,803
  12. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
38 more
  1. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  2. 2024Depth Anything V2cited by 1,422
  3. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  4. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  5. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  6. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  7. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  8. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  9. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  10. 2019Video Instance Segmentationcited by 393
  11. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  12. 2019Panoptic-DeepLabcited by 363
  13. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  14. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 231
  15. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  16. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  17. 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 174
  18. 2021Mask2Former for Video Instance Segmentationcited by 171
  19. 2018Video Object Segmentation with Language Referring Expressionscited by 162
  20. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  21. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  22. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  23. 2020STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videoscited by 121
  24. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 117
  25. 2021SeqFormer: Sequential Transformer for Video Instance Segmentationcited by 98
  26. 2022In Defense of Online Models for Video Instance Segmentationcited by 97
  27. 2020ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentationcited by 96
  28. 2022VITA: Video Instance Segmentation via Object Token Associationcited by 94
  29. 2022MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Trainingcited by 94
  30. 2022Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentationcited by 79
  31. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  32. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 75
  33. 2024Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentationcited by 71
  34. 2024Depth Any Video with Scalable Synthetic Datacited by 57
  35. 2022A Generalized Framework for Video Instance Segmentationcited by 53
  36. 2023CTVIS: Consistent Training for Online Video Instance Segmentationcited by 51
  37. 2023DVIS++: Improved Decoupled Framework for Universal Video Segmentationcited by 46
  38. 2021PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentationcited by 36

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VAcited by 7
  2. 2025SVAC: Scaling Is All You Need For Referring Video Object Segmentationcited by 1
  3. 2026SaSaSaSa2VA: 2nd Place of the 5th PVUW MeViS-Text Trackcited by 0