# Big Data application in congestion detection and classification using Apache spark

arXiv: https://arxiv.org/abs/2101.06524 · HTML: https://mlsanity.com/paper/2101.06524/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 18 references that are in the map.
- First posted on arXiv 16 Jan 2021.

## Builds on (most cited first, 2 of 2)

- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Improved Weighted Random Forest for Classification Problems](https://mlsanity.com/paper/2009.00534/) (2020)

## Built on by (most cited first, 1 of 1)

- [A data-driven personalized smart lighting recommender system](https://mlsanity.com/paper/2104.02164/) (2021)

## Abstract (arXiv)

With the era of big data, an explosive amount of information is now available. This enormous increase of Big Data in both academia and industry requires large-scale data processing systems. A large body of research is behind optimizing Spark's performance to make it state of the art, a fast and general data processing system. Many science and engineering fields have advanced with Big Data analytics, such as Biology, finance, and transportation. Intelligent transportation systems (ITS) gain popularity and direct benefit from the richness of information. The objective is to improve the safety and management of transportation networks by reducing congestion and incidents. The first step toward the goal is better understanding, modeling, and detecting congestion across a network efficiently and effectively. In this study, we introduce an efficient congestion detection model. The underlying network consists of 3017 segments in I-35, I-80, I-29, and I-380 freeways with an overall length of 1570 miles and averaged (0.4-0.6) miles per segment. The result of congestion detection shows the proposed method is 90% accurate while has reduced computation time by 99.88%.