# Spatio-Temporal Point Process for Multiple Object Tracking

arXiv: https://arxiv.org/abs/2302.02444 · HTML: https://mlsanity.com/paper/2302.02444/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 73 references that are in the map.
- First posted on arXiv 5 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking](https://mlsanity.com/paper/1504.01942/) (2015)
- [Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks](https://mlsanity.com/paper/1512.04143/) (2015)
- [Online Multi-Object Tracking with Dual Matching Attention Networks](https://mlsanity.com/paper/1902.00749/) (2019)
- [POI: Multiple Object Tracking with High Performance Detection and Appearance Feature](https://mlsanity.com/paper/1610.06136/) (2016)
- [Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism](https://mlsanity.com/paper/1708.02843/) (2017)
- [Recurrent Autoregressive Networks for Online Multi-Object Tracking](https://mlsanity.com/paper/1711.02741/) (2017)
- [Object detection via a multi-region & semantic segmentation-aware CNN model](https://mlsanity.com/paper/1505.01749/) (2015)
- [Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor](https://mlsanity.com/paper/1504.02340/) (2015)
- [Deep Network Flow for Multi-Object Tracking](https://mlsanity.com/paper/1706.08482/) (2017)
- [Multi-Person Tracking by Multicut and Deep Matching](https://mlsanity.com/paper/1608.05404/) (2016)
- [Multistage Campaigning in Social Networks](https://mlsanity.com/paper/1606.03816/) (2016)
- [A Tube-and-Droplet-based Approach for Representing and Analyzing Motion Trajectories](https://mlsanity.com/paper/1609.03058/) (2016)
- [A Hybrid Data Association Framework for Robust Online Multi-Object Tracking](https://mlsanity.com/paper/1703.10764/) (2017)
- [Joint Graph Decomposition and Node Labeling: Problem, Algorithms, Applications](https://mlsanity.com/paper/1611.04399/) (2016)
- [Machine Learning Methods for Data Association in Multi-Object Tracking](https://mlsanity.com/paper/1802.06897/) (2018)
- [Joint Learning of Siamese CNNs and Temporally Constrained Metrics for Tracklet Association](https://mlsanity.com/paper/1605.04502/) (2016)
- [SOT for MOT](https://mlsanity.com/paper/1712.01059/) (2017)
- [Multiple Target Tracking by Learning Feature Representation and Distance Metric Jointly](https://mlsanity.com/paper/1802.03252/) (2018)

## Built on by (most cited first, 5 of 5)

- [EGVD: Event-Guided Video Deraining](https://mlsanity.com/paper/2309.17239/) (2023)
- [Transformer with Koopman-Enhanced Graph Convolutional Network for Spatiotemporal Dynamics Forecasting](https://mlsanity.com/paper/2507.03855/) (2025)
- [Deep spatio-temporal point processes: Advances and new directions](https://mlsanity.com/paper/2504.06364/) (2025)
- [Marked Temporal Bayesian Flow Point Processes](https://mlsanity.com/paper/2410.19512/) (2024)
- [P-K-GCN: Physics-augmented Koopman-enhanced Graph Convolutional Network for Deep Spatiotemporal Super-resolution](https://mlsanity.com/paper/2606.19303/) (2026)

## Abstract (arXiv)

Multiple Object Tracking (MOT) focuses on modeling the relationship of detected objects among consecutive frames and merge them into different trajectories. MOT remains a challenging task as noisy and confusing detection results often hinder the final performance. Furthermore, most existing research are focusing on improving detection algorithms and association strategies. As such, we propose a novel framework that can effectively predict and mask-out the noisy and confusing detection results before associating the objects into trajectories. In particular, we formulate such "bad" detection results as a sequence of events and adopt the spatio-temporal point process}to model such events. Traditionally, the occurrence rate in a point process is characterized by an explicitly defined intensity function, which depends on the prior knowledge of some specific tasks. Thus, designing a proper model is expensive and time-consuming, with also limited ability to generalize well. To tackle this problem, we adopt the convolutional recurrent neural network (conv-RNN) to instantiate the point process, where its intensity function is automatically modeled by the training data. Furthermore, we show that our method captures both temporal and spatial evolution, which is essential in modeling events for MOT. Experimental results demonstrate notable improvements in addressing noisy and confusing detection results in MOT datasets. An improved state-of-the-art performance is achieved by incorporating our baseline MOT algorithm with the spatio-temporal point process model.