# SoDA: Multi-Object Tracking with Soft Data Association

arXiv: https://arxiv.org/abs/2008.07725 · HTML: https://mlsanity.com/paper/2008.07725/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Aug 2020.

## Built on by (most cited first, 12 of 12)

- [Wayformer: Motion Forecasting via Simple & Efficient Attention Networks](https://mlsanity.com/paper/2207.05844/) (2022)
- [Scene Transformer: A unified architecture for predicting multiple agent trajectories](https://mlsanity.com/paper/2106.08417/) (2021)
- [SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Models](https://mlsanity.com/paper/2411.13112/) (2024)
- [Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking](https://mlsanity.com/paper/2206.03666/) (2022)
- [Local Metrics for Multi-Object Tracking](https://mlsanity.com/paper/2104.02631/) (2021)
- [Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey](https://mlsanity.com/paper/2205.10766/) (2022)
- [Can Deep Learning be Applied to Model-Based Multi-Object Tracking?](https://mlsanity.com/paper/2202.07909/) (2022)
- [BOTT: Box Only Transformer Tracker for 3D Object Tracking](https://mlsanity.com/paper/2308.08753/) (2023)
- [STT: Stateful Tracking with Transformers for Autonomous Driving](https://mlsanity.com/paper/2405.00236/) (2024)
- [Tiny Neural Networks for Multi-Object Tracking in a Modular Kalman Framework](https://mlsanity.com/paper/2504.02519/) (2025)
- [An Approximate Dynamic Programming Framework for Occlusion-Robust Multi-Object Tracking](https://mlsanity.com/paper/2405.15137/) (2024)
- [The Progression of Transformers from Language to Vision to MOT: A Literature Review on Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2406.16784/) (2024)

## Abstract (arXiv)

Robust multi-object tracking (MOT) is a prerequisite fora safe deployment of self-driving cars. Tracking objects, however, remains a highly challenging problem, especially in cluttered autonomous driving scenes in which objects tend to interact with each other in complex ways and frequently get occluded. We propose a novel approach to MOT that uses attention to compute track embeddings that encode the spatiotemporal dependencies between observed objects. This attention measurement encoding allows our model to relax hard data associations, which may lead to unrecoverable errors. Instead, our model aggregates information from all object detections via soft data associations. The resulting latent space representation allows our model to learn to reason about occlusions in a holistic data-driven way and maintain track estimates for objects even when they are occluded. Our experimental results on the Waymo OpenDataset suggest that our approach leverages modern large-scale datasets and performs favorably compared to the state of the art in visual multi-object tracking.