# Leveraging Stereo-Camera Data for Real-Time Dynamic Obstacle Detection and Tracking

arXiv: https://arxiv.org/abs/2007.10743 · HTML: https://mlsanity.com/paper/2007.10743/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2020.

## Built on by (most cited first, 8 of 8)

- [Learning a State Representation and Navigation in Cluttered and Dynamic Environments](https://mlsanity.com/paper/2103.04351/) (2021)
- [A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with Dynamic Obstacle Trajectory Prediction and Its Application with LLMs](https://mlsanity.com/paper/2311.12893/) (2023)
- [Vision-aided UAV navigation and dynamic obstacle avoidance using gradient-based B-spline trajectory optimization](https://mlsanity.com/paper/2209.07003/) (2022)
- [Out of the Room: Generalizing Event-Based Dynamic Motion Segmentation for Complex Scenes](https://mlsanity.com/paper/2403.04562/) (2024)
- [StereoVoxelNet: Real-Time Obstacle Detection Based on Occupancy Voxels from a Stereo Camera Using Deep Neural Networks](https://mlsanity.com/paper/2209.08459/) (2022)
- [A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera](https://mlsanity.com/paper/2209.08258/) (2022)
- [Obstacle Identification and Ellipsoidal Decomposition for Fast Motion Planning in Unknown Dynamic Environments](https://mlsanity.com/paper/2209.14233/) (2022)
- [ODTFormer: Efficient Obstacle Detection and Tracking with Stereo Cameras Based on Transformer](https://mlsanity.com/paper/2403.14626/) (2024)

## Abstract (arXiv)

Dynamic obstacle avoidance is one crucial component for compliant navigation in crowded environments. In this paper we present a system for accurate and reliable detection and tracking of dynamic objects using noisy point cloud data generated by stereo cameras. Our solution is real-time capable and specifically designed for the deployment on computationally-constrained unmanned ground vehicles. The proposed approach identifies individual objects in the robot's surroundings and classifies them as either static or dynamic. The dynamic objects are labeled as either a person or a generic dynamic object. We then estimate their velocities to generate a 2D occupancy grid that is suitable for performing obstacle avoidance. We evaluate the system in indoor and outdoor scenarios and achieve real-time performance on a consumer-grade computer. On our test-dataset, we reach a MOTP of 0.07 ± 0.07m, and a MOTA of 85.3% for the detection and tracking of dynamic objects. We reach a precision of 96.9% for the detection of static objects.