mlsanityData to 14 Oct 2026

Enhanced Object Tracking by Self-Supervised Auxiliary Depth Estimation Learning

arXiv 2405.14195 ↗First posted 23 May 2024cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

RGB-D tracking significantly improves the accuracy of object tracking. However, its dependency on real depth inputs and the complexity involved in multi-modal fusion limit its applicability across various scenarios. The utilization of depth information in RGB-D tracking inspired us to propose a new method, named MDETrack, which trains a tracking network with an additional capability to understand the depth of scenes, through supervised or self-supervised auxiliary Monocular Depth Estimation learning. The outputs of MDETrack's unified feature extractor are fed to the side-by-side tracking head and auxiliary depth estimation head, respectively. The auxiliary module will be discarded in inference, thus keeping the same inference speed. We evaluated our models with various training strategies on multiple datasets, and the results show an improved tracking accuracy even without real depth. Through these findings we highlight the potential of depth estimation in enhancing object tracking performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 52 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

44 of 52 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2021Vision Transformers for Dense Predictioncited by 1,803
  4. 2015Spatial Transformer Networkscited by 1,429
  5. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  6. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  7. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  8. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  9. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  10. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 475
  11. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  12. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
32 more
  1. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 429
  2. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  3. 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 350
  4. 2018TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wildcited by 343
  5. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  6. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  7. 2023Visual Prompt Multi-Modal Trackingcited by 142
  8. 2019Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Drivingcited by 137
  9. 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
  10. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  11. 2018Auxiliary Tasks in Multi-task Learningcited by 108
  12. 2018Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understandingcited by 101
  13. 2022Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimationcited by 100
  14. 2019Self-Supervised Generalisation with Meta Auxiliary Learningcited by 88
  15. 2021Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detectioncited by 84
  16. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  17. 2020Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Drivingcited by 73
  18. 2021DepthTrack : Unveiling the Power of RGBD Trackingcited by 67
  19. 2021Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detectioncited by 67
  20. 2019Multi-Modal Fusion for End-to-End RGB-T Trackingcited by 63
  21. 2023Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Trackingcited by 53
  22. 2022RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimationcited by 36
  23. 2023Universal Few-shot Learning of Dense Prediction Tasks with Visual Token Matchingcited by 29
  24. 2024Depth-aware Test-Time Training for Zero-shot Video Object Segmentationcited by 27
  25. 2019DAL -- A Deep Depth-aware Long-term Trackercited by 26
  26. 2023RM-Depth: Unsupervised Learning of Recurrent Monocular Depth in Dynamic Scenescited by 22
  27. 2023SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimationcited by 21
  28. 2022Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamicscited by 20
  29. 2022Self-Supervised Monocular Depth Estimation: Solving the Edge-Fattening Problemcited by 20
  30. 2022RGBD Object Tracking: An In-depth Reviewcited by 11
  31. 2022Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Trackingcited by 9
  32. 2022Learning Dual-Fused Modality-Aware Representations for RGBD Trackingcited by 5

Built on by

Not cited yet.