# Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision

arXiv: https://arxiv.org/abs/2603.18298 · HTML: https://mlsanity.com/paper/2603.18298/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 49 references that are in the map.
- First posted on arXiv 18 Mar 2026.

## Builds on (most cited first, 30 of 37)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [M3D-RPN: Monocular 3D Region Proposal Network for Object Detection](https://mlsanity.com/paper/1907.06038/) (2019)
- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [CenterMask : Real-Time Anchor-Free Instance Segmentation](https://mlsanity.com/paper/1911.06667/) (2019)
- [SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking](https://mlsanity.com/paper/2111.09621/) (2021)
- [TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking](https://mlsanity.com/paper/2104.00194/) (2021)
- [MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries](https://mlsanity.com/paper/2205.00613/) (2022)
- [Monocular Quasi-Dense 3D Object Tracking](https://mlsanity.com/paper/2103.07351/) (2021)
- [DEFT: Detection Embeddings for Tracking](https://mlsanity.com/paper/2102.02267/) (2021)
- [Auto4D: Learning to Label 4D Objects from Sequential Point Clouds](https://mlsanity.com/paper/2101.06586/) (2021)
- [ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association](https://mlsanity.com/paper/2405.08909/) (2024)
- [Time3D: End-to-End Joint Monocular 3D Object Detection and Tracking for Autonomous Driving](https://mlsanity.com/paper/2205.14882/) (2022)
- [FANTrack: 3D Multi-Object Tracking with Feature Association Network](https://mlsanity.com/paper/1905.02843/) (2019)
- [3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention](https://mlsanity.com/paper/2203.10926/) (2022)
- [Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Tracking](https://mlsanity.com/paper/2101.12159/) (2021)
- [Multiple Object Tracking by Flowing and Fusing](https://mlsanity.com/paper/2001.11180/) (2020)
- [Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation](https://mlsanity.com/paper/2307.06125/) (2023)
- [Self-Supervised Tracking via Target-Aware Data Synthesis](https://mlsanity.com/paper/2106.10900/) (2021)
- [Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Tracking](https://mlsanity.com/paper/2510.10287/) (2025)

## Built on by (most cited first, 2 of 2)

- [Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Tracking](https://mlsanity.com/paper/2604.25405/) (2026)
- [GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure](https://mlsanity.com/paper/2608.14428/) (2026)

## Abstract (arXiv)

Monocular 3D object tracking aims to estimate temporally consistent 3D object poses across video frames, enabling autonomous agents to reason about scene dynamics. However, existing state-of-the-art approaches are fully supervised and rely on dense 3D annotations over long video sequences, which are expensive to obtain and difficult to scale. In this work, we address this fundamental limitation by proposing the first sparsely supervised framework for monocular 3D object tracking. Our approach decomposes the task into two sequential sub-problems: 2D query matching and 3D geometry estimation. Both components leverage the spatio-temporal consistency of image sequences to augment a sparse set of labeled samples and learn rich 2D and 3D representations of the scene. Leveraging these learned cues, our model automatically generates high-quality 3D pseudolabels across entire videos, effectively transforming sparse supervision into dense 3D track annotations. This enables existing fully-supervised trackers to effectively operate under extreme label sparsity. Extensive experiments on the KITTI and nuScenes datasets demonstrate that our method significantly improves tracking performance, achieving an improvement of up to 15.50 p.p. while using at most four ground truth annotations per track.