mlsanityData to 14 Oct 2026

Tracking Objects with 3D Representation from Videos

arXiv 2306.05416 ↗First posted 8 Jun 2023cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Data association is a knotty problem for 2D Multiple Object Tracking due to the object occlusion. However, in 3D space, data association is not so hard. Only with a 3D Kalman Filter, the online object tracker can associate the detections from LiDAR. In this paper, we rethink the data association in 2D MOT and utilize the 3D object representation to separate each object in the feature space. Unlike the existing depth-based MOT methods, the 3D object representation can be jointly learned with the object association module. Besides, the object's 3D representation is learned from the video and supervised by the 2D tracking labels without additional manual annotations from LiDAR or pretrained depth estimator. With 3D object representation learning from Pseudo 3D object labels in monocular videos, we propose a new 2D MOT paradigm, called P3DTrack. Extensive experiments show the effectiveness of our method. We achieve new state-of-the-art performance on the large-scale Waymo Open Dataset.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 58 of its 68 references that are in the map. First posted on arXiv 8 Jun 2023.

Builds on

58 of 68 references.

References · 58

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  5. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  7. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  8. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  10. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  11. 2019Objects as Pointscited by 1,175
  12. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
38 more
  1. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  2. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  3. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  4. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 786
  5. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  6. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  7. 2016Performance Measures and a Data Set for Multi-Target, Multi-Camera Trackingcited by 692
  8. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  9. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
  10. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 552
  11. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  12. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 490
  13. 2020Tracking Objects as Pointscited by 483
  14. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  15. 2017Deep Layer Aggregationcited by 436
  16. 2019Tracking without bells and whistlescited by 382
  17. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  18. 2022Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Trackingcited by 346
  19. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  20. 2020Consistent Video Depth Estimationcited by 220
  21. 2019Learning a Neural Solver for Multiple Object Trackingcited by 198
  22. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 185
  23. 2021Learning to Track with Object Permanencecited by 119
  24. 2022Global Tracking Transformerscited by 110
  25. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 91
  26. 2022BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereocited by 90
  27. 2022Fully Sparse 3D Object Detectioncited by 84
  28. 2021Monocular Quasi-Dense 3D Object Trackingcited by 81
  29. 2018Joint Monocular 3D Vehicle Detection and Trackingcited by 77
  30. 2019How To Train Your Deep Multi-Object Trackercited by 71
  31. 2020RetinaTrack: Online Single Stage Joint Detection and Trackingcited by 67
  32. 2020AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metricscited by 63
  33. 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Trackingcited by 61
  34. 2021DEFT: Detection Embeddings for Trackingcited by 56
  35. 2021Consistent Depth of Moving Objects in Videocited by 56
  36. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  37. 2021Learnable Graph Matching: Incorporating Graph Partitioning with Deep Feature Learning for Multiple Object Trackingcited by 47
  38. 2021Immortal Tracker: Tracklet Never Diescited by 42

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2025DexVLG: Dexterous Vision-Language-Grasp Model at Scalecited by 20
  2. 2023Propagating Semantic Labels in Video Datacited by 2