# Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking

arXiv: https://arxiv.org/abs/1802.09298 · HTML: https://mlsanity.com/paper/1802.09298/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2018.

## Built on by (most cited first, 30 of 40)

- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [EagerMOT: 3D Multi-Object Tracking via Sensor Fusion](https://mlsanity.com/paper/2104.14682/) (2021)
- [Monocular Quasi-Dense 3D Object Tracking](https://mlsanity.com/paper/2103.07351/) (2021)
- [Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation](https://mlsanity.com/paper/2204.04656/) (2022)
- [QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking](https://mlsanity.com/paper/2210.06984/) (2022)
- [OVTrack: Open-Vocabulary Multiple Object Tracking](https://mlsanity.com/paper/2304.08408/) (2023)
- [DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association](https://mlsanity.com/paper/2202.12100/) (2022)
- [CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusion](https://mlsanity.com/paper/2212.01247/) (2022)
- [Time3D: End-to-End Joint Monocular 3D Object Detection and Tracking for Autonomous Driving](https://mlsanity.com/paper/2205.14882/) (2022)
- [Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving](https://mlsanity.com/paper/2308.01471/) (2023)
- [PolarMOT: How Far Can Geometric Relations Take Us in 3D Multi-Object Tracking?](https://mlsanity.com/paper/2208.01957/) (2022)
- [TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM](https://mlsanity.com/paper/2210.16204/) (2022)
- [TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking](https://mlsanity.com/paper/2101.04206/) (2021)
- [MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusion](https://mlsanity.com/paper/2306.17000/) (2023)
- [DeNoising-MOT: Towards Multiple Object Tracking with Severe Occlusions](https://mlsanity.com/paper/2309.04682/) (2023)
- [QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving](https://mlsanity.com/paper/2404.01486/) (2024)
- [CAVIS: Context-Aware Video Instance Segmentation](https://mlsanity.com/paper/2407.03010/) (2024)
- [CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2311.00987/) (2023)
- [PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Search](https://mlsanity.com/paper/2403.15712/) (2024)
- [DirectTracker: 3D Multi-Object Tracking Using Direct Image Alignment and Photometric Bundle Adjustment](https://mlsanity.com/paper/2209.14965/) (2022)
- [Interactive Multi-scale Fusion of 2D and 3D Features for Multi-object Tracking](https://mlsanity.com/paper/2203.16268/) (2022)
- [ViewBirdiformer: Learning to recover ground-plane crowd trajectories and ego-motion from a single ego-centric view](https://mlsanity.com/paper/2210.06332/) (2022)
- [People Tracking in Panoramic Video for Guiding Robots](https://mlsanity.com/paper/2206.02735/) (2022)
- [Trajectory Poisson multi-Bernoulli mixture filter for traffic monitoring using a drone](https://mlsanity.com/paper/2306.16890/) (2023)
- [An End-to-End Framework of Road User Detection, Tracking, and Prediction from Monocular Images](https://mlsanity.com/paper/2308.05026/) (2023)
- [CrossTracker: Robust Multi-modal 3D Multi-Object Tracking via Cross Correction](https://mlsanity.com/paper/2411.18850/) (2024)
- [How Cars Move: Analyzing Driving Dynamics for Safer Urban Traffic](https://mlsanity.com/paper/2412.04020/) (2024)
- [FasterVideo: Efficient Online Joint Object Detection And Tracking](https://mlsanity.com/paper/2204.07394/) (2022)
- [A survey on deep learning approaches for data integration in autonomous driving system](https://mlsanity.com/paper/2306.11740/) (2023)
- [Runtime Stealthy Perception Attacks against DNN-based Adaptive Cruise Control Systems](https://mlsanity.com/paper/2307.08939/) (2023)

## Abstract (arXiv)

This paper introduces geometry and object shape and pose costs for multi-object tracking in urban driving scenarios. Using images from a monocular camera alone, we devise pairwise costs for object tracks, based on several 3D cues such as object pose, shape, and motion. The proposed costs are agnostic to the data association method and can be incorporated into any optimization framework to output the pairwise data associations. These costs are easy to implement, can be computed in real-time, and complement each other to account for possible errors in a tracking-by-detection framework. We perform an extensive analysis of the designed costs and empirically demonstrate consistent improvement over the state-of-the-art under varying conditions that employ a range of object detectors, exhibit a variety in camera and object motions, and, more importantly, are not reliant on the choice of the association framework. We also show that, by using the simplest of associations frameworks (two-frame Hungarian assignment), we surpass the state-of-the-art in multi-object-tracking on road scenes. More qualitative and quantitative results can be found at the following URL: https://junaidcs032.github.io/Geometry_ObjectShape_MOT/.