# A Vision-based System for Traffic Anomaly Detection using Deep Learning and Decision Trees

arXiv: https://arxiv.org/abs/2104.06856 · HTML: https://mlsanity.com/paper/2104.06856/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 32 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 2 of 2)

- [Anomaly Detection in Road Traffic Using Visual Surveillance: A Survey](https://mlsanity.com/paper/1901.08292/) (2019)
- [Artificial Intelligence Enabled Traffic Monitoring System](https://mlsanity.com/paper/2010.01217/) (2020)

## Built on by (most cited first, 26 of 26)

- [The 5th AI City Challenge](https://mlsanity.com/paper/2104.12233/) (2021)
- [TAD: A Large-Scale Benchmark for Traffic Accidents Detection from Video Surveillance](https://mlsanity.com/paper/2209.12386/) (2022)
- [Deep Video Anomaly Detection: Opportunities and Challenges](https://mlsanity.com/paper/2110.05086/) (2021)
- [Label-Free Multivariate Time Series Anomaly Detection](https://mlsanity.com/paper/2312.11549/) (2023)
- [Real-time Multi-Class Helmet Violation Detection Using Few-Shot Data Sampling Technique and YOLOv8](https://mlsanity.com/paper/2304.08256/) (2023)
- [MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection](https://mlsanity.com/paper/2510.21449/) (2025)
- [A Region-Based Deep Learning Approach to Automated Retail Checkout](https://mlsanity.com/paper/2204.08584/) (2022)
- [Enhancing Traffic Safety with Parallel Dense Video Captioning for End-to-End Event Analysis](https://mlsanity.com/paper/2404.08229/) (2024)
- [Deep Learning based Computer Vision Methods for Complex Traffic Environments Perception: A Review](https://mlsanity.com/paper/2211.05120/) (2022)
- [An Uncertainty-aware Loss Function for Training Neural Networks with Calibrated Predictions](https://mlsanity.com/paper/2110.03260/) (2021)
- [DeepSegmenter: Temporal Action Localization for Detecting Anomalies in Untrimmed Naturalistic Driving Videos](https://mlsanity.com/paper/2304.08261/) (2023)
- [Robust Roadside Perception: an Automated Data Synthesis Pipeline Minimizing Human Annotation](https://mlsanity.com/paper/2306.17302/) (2023)
- [A Survey on Video Anomaly Detection via Deep Learning: Human, Vehicle, and Environment](https://mlsanity.com/paper/2508.14203/) (2025)
- [AI-Based Framework for Understanding Car Following Behaviors of Drivers in A Naturalistic Driving Environment](https://mlsanity.com/paper/2301.09315/) (2023)
- [Low-Light Image Enhancement Framework for Improved Object Detection in Fisheye Lens Datasets](https://mlsanity.com/paper/2404.10078/) (2024)
- [CoT-VLM4Tar: Chain-of-Thought Guided Vision-Language Models for Traffic Anomaly Resolution](https://mlsanity.com/paper/2503.01632/) (2025)
- [Machine Learning-Based Classification Algorithms for the Prediction of Coronary Heart Diseases](https://mlsanity.com/paper/2112.01503/) (2021)
- [Classification of Anuran Frog Species Using Machine Learning](https://mlsanity.com/paper/2112.05148/) (2021)
- [Travel Time, Distance and Costs Optimization for Paratransit Operations using Graph Convolutional Neural Network](https://mlsanity.com/paper/2205.10507/) (2022)
- [Real-Time Helmet Violation Detection Using YOLOv5 and Ensemble Learning](https://mlsanity.com/paper/2304.09246/) (2023)
- [MSight: An Edge-Cloud Infrastructure-based Perception System for Connected Automated Vehicles](https://mlsanity.com/paper/2310.05290/) (2023)
- [uTRAND: Unsupervised Anomaly Detection in Traffic Trajectories](https://mlsanity.com/paper/2404.12712/) (2024)
- [Cerberus: Real-Time Video Anomaly Detection via Cascaded Vision-Language Models](https://mlsanity.com/paper/2510.16290/) (2025)
- [Rethinking the Detection Head Configuration for Traffic Object Detection](https://mlsanity.com/paper/2210.03883/) (2022)
- [SigSegment: A Signal-Based Segmentation Algorithm for Identifying Anomalous Driving Behaviours in Naturalistic Driving Videos](https://mlsanity.com/paper/2304.09247/) (2023)
- [STRIDE: Street View-based Environmental Feature Detection and Pedestrian Collision Prediction](https://mlsanity.com/paper/2308.13183/) (2023)

## Abstract (arXiv)

Any intelligent traffic monitoring system must be able to detect anomalies such as traffic accidents in real time. In this paper, we propose a Decision-Tree - enabled approach powered by Deep Learning for extracting anomalies from traffic cameras while accurately estimating the start and end time of the anomalous event. Our approach included creating a detection model, followed by anomaly detection and analysis. YOLOv5 served as the foundation for our detection model. The anomaly detection and analysis step entail traffic scene background estimation, road mask extraction, and adaptive thresholding. Candidate anomalies were passed through a decision tree to detect and analyze final anomalies. The proposed approach yielded an F1 score of 0.8571, and an S4 score of 0.5686, per the experimental validation.