# Measuring the Confidence of Traffic Forecasting Models: Techniques, Experimental Comparison and Guidelines towards Their Actionability

arXiv: https://arxiv.org/abs/2210.16049 · HTML: https://mlsanity.com/paper/2210.16049/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 89 references that are in the map.
- First posted on arXiv 28 Oct 2022.

## Builds on (most cited first, 19 of 19)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Graph Neural Network for Traffic Forecasting: A Survey](https://mlsanity.com/paper/2101.11174/) (2021)
- [Learning by Transduction](https://mlsanity.com/paper/1301.7375/) (2013)
- [Towards Bayesian Deep Learning: A Framework and Some Existing Methods](https://mlsanity.com/paper/1608.06884/) (2016)
- [Active Learning Using Uncertainty Information](https://mlsanity.com/paper/1702.08540/) (2017)
- [Uncertainty-Aware Driver Trajectory Prediction at Urban Intersections](https://mlsanity.com/paper/1901.05105/) (2019)
- [Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AI](https://mlsanity.com/paper/2106.01410/) (2021)
- [Detecting Concept Drift With Neural Network Model Uncertainty](https://mlsanity.com/paper/2107.01873/) (2021)
- [Deep Learning for Road Traffic Forecasting: Does it Make a Difference?](https://mlsanity.com/paper/2012.02260/) (2020)
- [DROGON: A Trajectory Prediction Model based on Intention-Conditioned Behavior Reasoning](https://mlsanity.com/paper/1908.00024/) (2019)
- [From Data to Actions in Intelligent Transportation Systems: a Prescription of Functional Requirements for Model Actionability](https://mlsanity.com/paper/2002.02210/) (2020)
- [Efficient Uncertainty-aware Decision-making for Automated Driving Using Guided Branching](https://mlsanity.com/paper/2003.02746/) (2020)
- [Heteroscedastic Gaussian processes for uncertainty modeling in large-scale crowdsourced traffic data](https://mlsanity.com/paper/1812.08733/) (2018)
- [Short-term bus travel time prediction for transfer synchronization with intelligent uncertainty handling](https://mlsanity.com/paper/2104.06819/) (2021)
- [Coordination and Trajectory Prediction for Vehicle Interactions via Bayesian Generative Modeling](https://mlsanity.com/paper/1905.00587/) (2019)
- [Transfer Learning and Online Learning for Traffic Forecasting under Different Data Availability Conditions: Alternatives and Pitfalls](https://mlsanity.com/paper/2005.05069/) (2020)

## Built on by (most cited first, 1 of 1)

- [Uncertainty Quantification for Image-based Traffic Prediction across Cities](https://mlsanity.com/paper/2308.06129/) (2023)

## Abstract (arXiv)

The estimation of the amount of uncertainty featured by predictive machine learning models has acquired a great momentum in recent years. Uncertainty estimation provides the user with augmented information about the model's confidence in its predicted outcome. Despite the inherent utility of this information for the trustworthiness of the user, there is a thin consensus around the different types of uncertainty that one can gauge in machine learning models and the suitability of different techniques that can be used to quantify the uncertainty of a specific model. This subject is mostly non existent within the traffic modeling domain, even though the measurement of the confidence associated to traffic forecasts can favor significantly their actionability in practical traffic management systems. This work aims to cover this lack of research by reviewing different techniques and metrics of uncertainty available in the literature, and by critically discussing how confidence levels computed for traffic forecasting models can be helpful for researchers and practitioners working in this research area. To shed light with empirical evidence, this critical discussion is further informed by experimental results produced by different uncertainty estimation techniques over real traffic data collected in Madrid (Spain), rendering a general overview of the benefits and caveats of every technique, how they can be compared to each other, and how the measured uncertainty decreases depending on the amount, quality and diversity of data used to produce the forecasts.