# Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1602.00991 · HTML: https://mlsanity.com/paper/1602.00991/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2016.

## Built on by (most cited first, 13 of 13)

- [TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking](https://mlsanity.com/paper/2104.00194/) (2021)
- [GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021](https://mlsanity.com/paper/2202.11983/) (2022)
- [MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking](https://mlsanity.com/paper/2107.10433/) (2021)
- [Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning](https://mlsanity.com/paper/2205.03212/) (2022)
- [LOPR: Latent Occupancy PRediction using Generative Models](https://mlsanity.com/paper/2210.01249/) (2022)
- [AutoGraph: Predicting Lane Graphs from Traffic Observations](https://mlsanity.com/paper/2306.15410/) (2023)
- [People Tracking in Panoramic Video for Guiding Robots](https://mlsanity.com/paper/2206.02735/) (2022)
- [LOST-3DSG: Lightweight Open-Vocabulary 3D Scene Graphs with Semantic Tracking in Dynamic Environments](https://mlsanity.com/paper/2601.02905/) (2026)
- [PAT: Pseudo-Adversarial Training For Detecting Adversarial Videos](https://mlsanity.com/paper/2109.05695/) (2021)
- [Circular Accessible Depth: A Robust Traversability Representation for UGV Navigation](https://mlsanity.com/paper/2212.13676/) (2022)
- [Fast and Resource-Efficient Object Tracking on Edge Devices: A Measurement Study](https://mlsanity.com/paper/2309.02666/) (2023)
- [Finding a Needle in a Haystack: Tiny Flying Object Detection in 4K Videos using a Joint Detection-and-Tracking Approach](https://mlsanity.com/paper/2105.08253/) (2021)
- [CORT: Class-Oriented Real-time Tracking for Embedded Systems](https://mlsanity.com/paper/2407.17521/) (2024)

## Abstract (arXiv)

This paper presents to the best of our knowledge the first end-to-end object tracking approach which directly maps from raw sensor input to object tracks in sensor space without requiring any feature engineering or system identification in the form of plant or sensor models. Specifically, our system accepts a stream of raw sensor data at one end and, in real-time, produces an estimate of the entire environment state at the output including even occluded objects. We achieve this by framing the problem as a deep learning task and exploit sequence models in the form of recurrent neural networks to learn a mapping from sensor measurements to object tracks. In particular, we propose a learning method based on a form of input dropout which allows learning in an unsupervised manner, only based on raw, occluded sensor data without access to ground-truth annotations. We demonstrate our approach using a synthetic dataset designed to mimic the task of tracking objects in 2D laser data -- as commonly encountered in robotics applications -- and show that it learns to track many dynamic objects despite occlusions and the presence of sensor noise.