Online Multi-Target Tracking Using Recurrent Neural Networks
Not on the map. Why?
Abstract (arXiv)
We present a novel approach to online multi-target tracking based on recurrent neural networks (RNNs). Tracking multiple objects in real-world scenes involves many challenges, including a) an a-priori unknown and time-varying number of targets, b) a continuous state estimation of all present targets, and c) a discrete combinatorial problem of data association. Most previous methods involve complex models that require tedious tuning of parameters. Here, we propose for the first time, an end-to-end learning approach for online multi-target tracking. Existing deep learning methods are not designed for the above challenges and cannot be trivially applied to the task. Our solution addresses all of the above points in a principled way. Experiments on both synthetic and real data show promising results obtained at ~300 Hz on a standard CPU, and pave the way towards future research in this direction.
Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Apr 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 14; 2022: 19; 2023: 8; 2024: 7; 2025: 9; 2026: 2.
59 papers.
Sort:
- 2021Learning to Track with Object Permanencecited by 117
- 2021TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Trackingcited by 93
- 2022Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Spacecited by 66
- 2022Unifying Short and Long-Term Tracking with Graph Hierarchiescited by 62
- 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Trackingcited by 61
- 2023OVTrack: Open-Vocabulary Multiple Object Trackingcited by 53
- 2022GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021cited by 46
- 2022Tracking Every Thing in the Wildcited by 44
- 2021TransCenter: Transformers with Dense Representations for Multiple-Object Trackingcited by 42
- 2021Deep Learning for UAV-based Object Detection and Tracking: A Surveycited by 33
- 2021Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Trackingcited by 22
- 2023MotionTrack: Learning Motion Predictor for Multiple Object Trackingcited by 20
38 more
- 2021Split and Connect: A Universal Tracklet Booster for Multi-Object Trackingcited by 17
- 2023FOLT: Fast Multiple Object Tracking from UAV-captured Videos Based on Optical Flowcited by 15
- 2021Online Multiple Object Tracking with Cross-Task Synergycited by 12
- 2022In Pursuit of Many: A Review of Modern Multiple Object Tracking Systemscited by 12
- 2022Learning of Global Objective for Network Flow in Multi-Object Trackingcited by 11
- 2023Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planningcited by 9
- 2024ETTrack: Enhanced Temporal Motion Predictor for Multi-Object Trackingcited by 9
- 2021Multiple Object Tracking with Mixture Density Networks for Trajectory Estimationcited by 8
- 2022Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encodingcited by 8
- 2022Scenario-Assisted Deep Reinforcement Learningcited by 8
- 2022Recent Advances in Embedding Methods for Multi-Object Tracking: A Surveycited by 8
- 2021Safety-Oriented Pedestrian Motion and Scene Occupancy Forecastingcited by 6
- 2022Towards Frame Rate Agnostic Multi-Object Trackingcited by 6
- 2022Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detectioncited by 6
- 2022Can Deep Learning be Applied to Model-Based Multi-Object Tracking?cited by 5
- 2024Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensorscited by 5
- 2021Simple online and real-time tracking with occlusion handlingcited by 4
- 2023Transformer-Based Multi-Object Smoothing with Decoupled Data Association and Smoothingcited by 3
- 2025Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Modelscited by 3
- 2025NOOUGAT: Towards Unified Online and Offline Multi-Object Trackingcited by 3
- 2025One Graph to Track Them All: Dynamic GNNs for Single- and Multi-View Trackingcited by 2
- 2021Vehicle Detection and Tracking From Surveillance Cameras in Urban Scenescited by 1
- 2022Spatial-Temporal Map Vehicle Trajectory Detection Using Dynamic Mode Decomposition and Res-UNet+ Neural Networkscited by 1
- 2022Strong-TransCenter: Improved Multi-Object Tracking based on Transformers with Dense Representationscited by 1
- 2022Graph-Based Multi-Camera Soccer Player Trackercited by 1
- 2022Joint Counting, Detection and Re-Identification for Multi-Object Trackingcited by 1
- 2023Fast and Resource-Efficient Object Tracking on Edge Devices: A Measurement Studycited by 1
- 2023Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separationcited by 1
- 2024MAML MOT: Multiple Object Tracking based on Meta-Learningcited by 1
- 2024OCTrack: Benchmarking the Open-Corpus Multi-Object Trackingcited by 1
- 2025Tiny Neural Networks for Multi-Object Tracking in a Modular Kalman Frameworkcited by 1
- 2025A New Perspective On AI Safety Through Control Theory Methodologiescited by 1
- 2021Finding a Needle in a Haystack: Tiny Flying Object Detection in 4K Videos using a Joint Detection-and-Tracking Approachcited by 0
- 2021A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Datacited by 0
- 2021Multi-target tracking for video surveillance using deep affinity network: a brief reviewcited by 0
- 2022Unsupervised Multiple-Object Tracking with a Dynamical Variational Autoencodercited by 0
- 2022Sparse Message Passing Network with Feature Integration for Online Multiple Object Trackingcited by 0
- 2023Deep Learning Method for Cell-Wise Object Tracking, Velocity Estimation and Projection of Sensor Data over Timecited by 0
50 most cited of 59 All 59 (JSON)
- 2026Time-series Meets Complex Motion Modeling: Robust and Computational-effective Motion Predictor for Multi-object Tracking1 May 2026
- 2026Extend Your Horizon: A Device-Agnostic Surgical Tool Tracking Framework with Multi-View Optimization for Augmented Reality9 Mar 2026
- 2025StableTrack: Stabilizing Multi-Object Tracking on Low-Frequency Detections25 Nov 2025
- 2025Multi-Objective textitmin-max Online Convex Optimization15 Oct 2025
- 2025NOOUGAT: Towards Unified Online and Offline Multi-Object Tracking2 Sep 2025
- 2025One Graph to Track Them All: Dynamic GNNs for Single- and Multi-View Tracking11 Jul 2025
- 2025A New Perspective On AI Safety Through Control Theory Methodologies30 Jun 2025
- 2025Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models22 Jun 2025
- 2025Accurate Tracking of Arabidopsis Root Cortex Cell Nuclei in 3D Time-Lapse Microscopy Images Based on Genetic Algorithm17 Apr 2025
- 2025Tiny Neural Networks for Multi-Object Tracking in a Modular Kalman Framework3 Apr 2025
- 2025Stone Soup Multi-Target Tracking Feature Extraction For Autonomous Search And Track In Deep Reinforcement Learning Environment3 Mar 2025
- 2024CORT: Class-Oriented Real-time Tracking for Embedded Systems20 Jul 2024
38 more
- 2024OCTrack: Benchmarking the Open-Corpus Multi-Object Tracking19 Jul 2024
- 2024Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors10 Jul 2024
- 2024SSP-GNN: Learning to Track via Bilevel Optimization5 Jul 2024
- 2024ETTrack: Enhanced Temporal Motion Predictor for Multi-Object Tracking24 May 2024
- 2024MAML MOT: Multiple Object Tracking based on Meta-Learning12 May 2024
- 2024Ego-Motion Aware Target Prediction Module for Robust Multi-Object Tracking3 Apr 2024
- 2023Transformer-Based Multi-Object Smoothing with Decoupled Data Association and Smoothing22 Dec 2023
- 2023Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separation7 Dec 2023
- 2023Fast and Resource-Efficient Object Tracking on Edge Devices: A Measurement Study6 Sep 2023
- 2023FOLT: Fast Multiple Object Tracking from UAV-captured Videos Based on Optical Flow14 Aug 2023
- 2023Deep Learning Method for Cell-Wise Object Tracking, Velocity Estimation and Projection of Sensor Data over Time8 Jun 2023
- 2023MotionTrack: Learning Motion Predictor for Multiple Object Tracking5 Jun 2023
- 2023Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planning3 Jun 2023
- 2023OVTrack: Open-Vocabulary Multiple Object Tracking17 Apr 2023
- 2022Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detection20 Dec 2022
- 2022Joint Counting, Detection and Re-Identification for Multi-Object Tracking12 Dec 2022
- 2022Unifying Short and Long-Term Tracking with Graph Hierarchies6 Dec 2022
- 2022Sparse Message Passing Network with Feature Integration for Online Multiple Object Tracking6 Dec 2022
- 2022Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Space24 Nov 2022
- 2022Graph-Based Multi-Camera Soccer Player Tracker3 Nov 2022
- 2022Strong-TransCenter: Improved Multi-Object Tracking based on Transformers with Dense Representations24 Oct 2022
- 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking12 Oct 2022
- 2022Towards Frame Rate Agnostic Multi-Object Tracking23 Sep 2022
- 2022In Pursuit of Many: A Review of Modern Multiple Object Tracking Systems11 Sep 2022
- 2022Tracking Every Thing in the Wild26 Jul 2022
- 2022Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey22 May 2022
- 2022Learning of Global Objective for Network Flow in Multi-Object Tracking30 Mar 2022
- 2022GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 202124 Feb 2022
- 2022Unsupervised Multiple-Object Tracking with a Dynamical Variational Autoencoder18 Feb 2022
- 2022Can Deep Learning be Applied to Model-Based Multi-Object Tracking?16 Feb 2022
- 2022Scenario-Assisted Deep Reinforcement Learning9 Feb 2022
- 2022Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encoding3 Feb 2022
- 2022Spatial-Temporal Map Vehicle Trajectory Detection Using Dynamic Mode Decomposition and Res-UNet+ Neural Networks13 Jan 2022
- 2021Multi-target tracking for video surveillance using deep affinity network: a brief review29 Oct 2021
- 2021Deep Learning for UAV-based Object Detection and Tracking: A Survey25 Oct 2021
- 2021Vehicle Detection and Tracking From Surveillance Cameras in Urban Scenes25 Sep 2021
- 2021A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data27 Aug 2021
- 2021Multiple Object Tracking with Mixture Density Networks for Trajectory Estimation21 Jun 2021
50 newest of 59 All 59 (JSON)
- 2021Learning to Track with Object Permanencecited by 117
- 2021TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Trackingcited by 93
- 2022Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Spacecited by 66
- 2022Unifying Short and Long-Term Tracking with Graph Hierarchiescited by 62
- 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Trackingcited by 61
- 2023OVTrack: Open-Vocabulary Multiple Object Trackingcited by 53
- 2022GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021cited by 46
- 2022Tracking Every Thing in the Wildcited by 44
- 2021TransCenter: Transformers with Dense Representations for Multiple-Object Trackingcited by 42
- 2021Deep Learning for UAV-based Object Detection and Tracking: A Surveycited by 33
- 2021Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Trackingcited by 22
- 2023MotionTrack: Learning Motion Predictor for Multiple Object Trackingcited by 20
12 most cited of 59 All 59 (JSON)
- 2026Time-series Meets Complex Motion Modeling: Robust and Computational-effective Motion Predictor for Multi-object Tracking1 May 2026
- 2026Extend Your Horizon: A Device-Agnostic Surgical Tool Tracking Framework with Multi-View Optimization for Augmented Reality9 Mar 2026
- 2025StableTrack: Stabilizing Multi-Object Tracking on Low-Frequency Detections25 Nov 2025
- 2025Multi-Objective textitmin-max Online Convex Optimization15 Oct 2025
- 2025NOOUGAT: Towards Unified Online and Offline Multi-Object Tracking2 Sep 2025
- 2025One Graph to Track Them All: Dynamic GNNs for Single- and Multi-View Tracking11 Jul 2025
- 2025A New Perspective On AI Safety Through Control Theory Methodologies30 Jun 2025
- 2025Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models22 Jun 2025
- 2025Accurate Tracking of Arabidopsis Root Cortex Cell Nuclei in 3D Time-Lapse Microscopy Images Based on Genetic Algorithm17 Apr 2025
- 2025Tiny Neural Networks for Multi-Object Tracking in a Modular Kalman Framework3 Apr 2025
- 2025Stone Soup Multi-Target Tracking Feature Extraction For Autonomous Search And Track In Deep Reinforcement Learning Environment3 Mar 2025
- 2024CORT: Class-Oriented Real-time Tracking for Embedded Systems20 Jul 2024
12 newest of 59 All 59 (JSON)