# A Unified Object Motion and Affinity Model for Online Multi-Object Tracking

arXiv: https://arxiv.org/abs/2003.11291 · HTML: https://mlsanity.com/paper/2003.11291/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2020.

## Built on by (most cited first, 30 of 33)

- [Track to Detect and Segment: An Online Multi-Object Tracker](https://mlsanity.com/paper/2103.08808/) (2021)
- [MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?](https://mlsanity.com/paper/2108.09518/) (2021)
- [CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion](https://mlsanity.com/paper/2209.02540/) (2022)
- [Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook](https://mlsanity.com/paper/2112.02838/) (2021)
- [Simple Cues Lead to a Strong Multi-Object Tracker](https://mlsanity.com/paper/2206.04656/) (2022)
- [TransCenter: Transformers with Dense Representations for Multiple-Object Tracking](https://mlsanity.com/paper/2103.15145/) (2021)
- [AnimalTrack: A Benchmark for Multi-Animal Tracking in the Wild](https://mlsanity.com/paper/2205.00158/) (2022)
- [DyGLIP: A Dynamic Graph Model with Link Prediction for Accurate Multi-Camera Multiple Object Tracking](https://mlsanity.com/paper/2106.06856/) (2021)
- [One More Check: Making "Fake Background" Be Tracked Again](https://mlsanity.com/paper/2104.09441/) (2021)
- [Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.14829/) (2021)
- [Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking](https://mlsanity.com/paper/2105.02426/) (2021)
- [Online Multiple Object Tracking with Cross-Task Synergy](https://mlsanity.com/paper/2104.00380/) (2021)
- [Learning of Global Objective for Network Flow in Multi-Object Tracking](https://mlsanity.com/paper/2203.16210/) (2022)
- [Recurrent Aligned Network for Generalized Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2403.05810/) (2024)
- [Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey](https://mlsanity.com/paper/2205.10766/) (2022)
- [Depth Perspective-aware Multiple Object Tracking](https://mlsanity.com/paper/2207.04551/) (2022)
- [Enhancing Object Detection for Autonomous Driving by Optimizing Anchor Generation and Addressing Class Imbalance](https://mlsanity.com/paper/2104.03888/) (2021)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception](https://mlsanity.com/paper/2407.10876/) (2024)
- [BACTrack: Building Appearance Collection for Aerial Tracking](https://mlsanity.com/paper/2312.06136/) (2023)
- [MOTCOM: The Multi-Object Tracking Dataset Complexity Metric](https://mlsanity.com/paper/2207.10031/) (2022)
- [DSRRTracker: Dynamic Search Region Refinement for Attention-based Siamese Multi-Object Tracking](https://mlsanity.com/paper/2203.10729/) (2022)
- [Transformer-based assignment decision network for multiple object tracking](https://mlsanity.com/paper/2208.03571/) (2022)
- [Detection-aware multi-object tracking evaluation](https://mlsanity.com/paper/2212.08536/) (2022)
- [UTOPIA: Unconstrained Tracking Objects without Preliminary Examination via Cross-Domain Adaptation](https://mlsanity.com/paper/2306.09613/) (2023)
- [Temporal Correlation Meets Embedding: Towards a 2nd Generation of JDE-based Real-Time Multi-Object Tracking](https://mlsanity.com/paper/2407.14086/) (2024)
- [MOTS R-CNN: Cosine-margin-triplet loss for multi-object tracking](https://mlsanity.com/paper/2102.03512/) (2021)
- [STURE: Spatial-Temporal Mutual Representation Learning for Robust Data Association in Online Multi-Object Tracking](https://mlsanity.com/paper/2201.06824/) (2022)
- [Real-Time Siamese Multiple Object Tracker with Enhanced Proposals](https://mlsanity.com/paper/2202.04966/) (2022)
- [SCTracker: Multi-object tracking with shape and confidence constraints](https://mlsanity.com/paper/2305.09523/) (2023)

## Abstract (arXiv)

Current popular online multi-object tracking (MOT) solutions apply single object trackers (SOTs) to capture object motions, while often requiring an extra affinity network to associate objects, especially for the occluded ones. This brings extra computational overhead due to repetitive feature extraction for SOT and affinity computation. Meanwhile, the model size of the sophisticated affinity network is usually non-trivial. In this paper, we propose a novel MOT framework that unifies object motion and affinity model into a single network, named UMA, in order to learn a compact feature that is discriminative for both object motion and affinity measure. In particular, UMA integrates single object tracking and metric learning into a unified triplet network by means of multi-task learning. Such design brings advantages of improved computation efficiency, low memory requirement and simplified training procedure. In addition, we equip our model with a task-specific attention module, which is used to boost task-aware feature learning. The proposed UMA can be easily trained end-to-end, and is elegant - requiring only one training stage. Experimental results show that it achieves promising performance on several MOT Challenge benchmarks.