# TAU: A Framework for Video-Based Traffic Analytics Leveraging Artificial Intelligence and Unmanned Aerial Systems

arXiv: https://arxiv.org/abs/2303.00337 · HTML: https://mlsanity.com/paper/2303.00337/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 64 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Deep Learning in Video Multi-Object Tracking: A Survey](https://mlsanity.com/paper/1907.12740/) (2019)
- [A Survey on Instance Segmentation: State of the art](https://mlsanity.com/paper/2007.00047/) (2020)
- [Unsupervised Domain Adaptation using Generative Adversarial Networks for Semantic Segmentation of Aerial Images](https://mlsanity.com/paper/1905.03198/) (2019)
- [Randomly Initialized Convolutional Neural Network for the Recognition of COVID-19 using X-ray Images](https://mlsanity.com/paper/2105.08199/) (2021)
- [A Deep Learning-based Approach for Real-time Facemask Detection](https://mlsanity.com/paper/2110.08732/) (2021)
- [Activity Monitoring of Islamic Prayer (Salat) Postures using Deep Learning](https://mlsanity.com/paper/1911.04102/) (2019)
- [Parking Analytics Framework using Deep Learning](https://mlsanity.com/paper/2203.07792/) (2022)

## Built on by (most cited first, 3 of 3)

- [License Plate Super-Resolution Using Diffusion Models](https://mlsanity.com/paper/2309.12506/) (2023)
- [Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisons](https://mlsanity.com/paper/2311.07955/) (2023)
- [FMRFT: Fusion Mamba and DETR for Query Time Sequence Intersection Fish Tracking](https://mlsanity.com/paper/2409.01148/) (2024)

## Abstract (arXiv)

Smart traffic engineering and intelligent transportation services are in increasing demand from governmental authorities to optimize traffic performance and thus reduce energy costs, increase the drivers' safety and comfort, ensure traffic laws enforcement, and detect traffic violations. In this paper, we address this challenge, and we leverage the use of Artificial Intelligence (AI) and Unmanned Aerial Vehicles (UAVs) to develop an AI-integrated video analytics framework, called TAU (Traffic Analysis from UAVs), for automated traffic analytics and understanding. Unlike previous works on traffic video analytics, we propose an automated object detection and tracking pipeline from video processing to advanced traffic understanding using high-resolution UAV images. TAU combines six main contributions. First, it proposes a pre-processing algorithm to adapt the high-resolution UAV image as input to the object detector without lowering the resolution. This ensures an excellent detection accuracy from high-quality features, particularly the small size of detected objects from UAV images. Second, it introduces an algorithm for recalibrating the vehicle coordinates to ensure that vehicles are uniquely identified and tracked across the multiple crops of the same frame. Third, it presents a speed calculation algorithm based on accumulating information from successive frames. Fourth, TAU counts the number of vehicles per traffic zone based on the Ray Tracing algorithm. Fifth, TAU has a fully independent algorithm for crossroad arbitration based on the data gathered from the different zones surrounding it. Sixth, TAU introduces a set of algorithms for extracting twenty-four types of insights from the raw data collected. The code is shared here: https://github.com/bilel-bj/TAU. Video demonstrations are provided here: https://youtu.be/wXJV0H7LviU and here: https://youtu.be/kGv0gmtVEbI.