# Detecting Invisible People

arXiv: https://arxiv.org/abs/2012.08419 · HTML: https://mlsanity.com/paper/2012.08419/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2020.

## Built on by (most cited first, 30 of 31)

- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022)
- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)
- [STEP: Segmenting and Tracking Every Pixel](https://mlsanity.com/paper/2102.11859/) (2021)
- [MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking](https://mlsanity.com/paper/2303.10404/) (2023)
- [UCMCTrack: Multi-Object Tracking with Uniform Camera Motion Compensation](https://mlsanity.com/paper/2312.08952/) (2023)
- [PACE: Human and Camera Motion Estimation from in-the-wild Videos](https://mlsanity.com/paper/2310.13768/) (2023)
- [Quo Vadis: Is Trajectory Forecasting the Key Towards Long-Term Multi-Object Tracking?](https://mlsanity.com/paper/2210.07681/) (2022)
- [Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mind](https://mlsanity.com/paper/2404.05072/) (2024)
- [HMP: Hand Motion Priors for Pose and Shape Estimation from Video](https://mlsanity.com/paper/2312.16737/) (2023)
- [Towards Generalizable Multi-Object Tracking](https://mlsanity.com/paper/2406.00429/) (2024)
- [Revealing Occlusions with 4D Neural Fields](https://mlsanity.com/paper/2204.10916/) (2022)
- [DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes](https://mlsanity.com/paper/2302.07676/) (2023)
- [Using Diffusion Priors for Video Amodal Segmentation](https://mlsanity.com/paper/2412.04623/) (2024)
- [Tracking People by Predicting 3D Appearance, Location & Pose](https://mlsanity.com/paper/2112.04477/) (2021)
- [PoseBERT: A Generic Transformer Module for Temporal 3D Human Modeling](https://mlsanity.com/paper/2208.10211/) (2022)
- [Tracking through Containers and Occluders in the Wild](https://mlsanity.com/paper/2305.03052/) (2023)
- [COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation](https://mlsanity.com/paper/2408.16426/) (2024)
- [Video-based Sequential Bayesian Homography Estimation for Soccer Field Registration](https://mlsanity.com/paper/2311.10361/) (2023)
- [TAO-Amodal: A Benchmark for Tracking Any Object Amodally](https://mlsanity.com/paper/2312.12433/) (2023)
- [CAMOT: Camera Angle-aware Multi-Object Tracking](https://mlsanity.com/paper/2409.17533/) (2024)
- [Tracking Objects with 3D Representation from Videos](https://mlsanity.com/paper/2306.05416/) (2023)
- [SFSORT: Scene Features-based Simple Online Real-Time Tracker](https://mlsanity.com/paper/2404.07553/) (2024)
- [History-Aware Transformation of ReID Features for Multiple Object Tracking](https://mlsanity.com/paper/2503.12562/) (2025)
- [Predicting Long-horizon Futures by Conditioning on Geometry and Time](https://mlsanity.com/paper/2404.11554/) (2024)
- [One Homography is All You Need: IMM-based Joint Homography and Multiple Object State Estimation](https://mlsanity.com/paper/2409.02562/) (2024)
- [GRASPTrack: Geometry-Reasoned Association via Segmentation and Projection for Multi-Object Tracking](https://mlsanity.com/paper/2508.08117/) (2025)
- [t-RAIN: Robust generalization under weather-aliasing label shift attacks](https://mlsanity.com/paper/2305.08302/) (2023)
- [SMTrack: End-to-End Trained Spiking Neural Networks for Multi-Object Tracking in RGB Videos](https://mlsanity.com/paper/2508.14607/) (2025)

## Abstract (arXiv)

Monocular object detection and tracking have improved drastically in recent years, but rely on a key assumption: that objects are visible to the camera. Many offline tracking approaches reason about occluded objects post-hoc, by linking together tracklets after the object re-appears, making use of reidentification (ReID). However, online tracking in embodied robotic agents (such as a self-driving vehicle) fundamentally requires object permanence, which is the ability to reason about occluded objects before they re-appear. In this work, we re-purpose tracking benchmarks and propose new metrics for the task of detecting invisible objects, focusing on the illustrative case of people. We demonstrate that current detection and tracking systems perform dramatically worse on this task. We introduce two key innovations to recover much of this performance drop. We treat occluded object detection in temporal sequences as a short-term forecasting challenge, bringing to bear tools from dynamic sequence prediction. Second, we build dynamic models that explicitly reason in 3D, making use of observations produced by state-of-the-art monocular depth estimation networks. To our knowledge, ours is the first work to demonstrate the effectiveness of monocular depth estimation for the task of tracking and detecting occluded objects. Our approach strongly improves by 11.4% over the baseline in ablations and by 5.0% over the state-of-the-art in F1 score.