# Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty

arXiv: https://arxiv.org/abs/2211.02924 · HTML: https://mlsanity.com/paper/2211.02924/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 5 Nov 2022.

## Builds on (most cited first, 4 of 4)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [An Overview of 5G Advanced Evolution in 3GPP Release 18](https://mlsanity.com/paper/2201.01358/) (2022)
- [A Convolutional Attention Based Deep Network Solution for UAV Network Attack Recognition over Fading Channels and Interference](https://mlsanity.com/paper/2207.10810/) (2022)

## Built on by (most cited first, 1 of 1)

- [PCA-Featured Transformer for Jamming Detection in 5G UAV Networks](https://mlsanity.com/paper/2412.15312/) (2024)

## Abstract (arXiv)

Only increasing accuracy without considering uncertainty may negatively impact Deep Neural Network (DNN) decision-making and decrease its reliability. This paper proposes five combined preprocessing and post-processing methods for time-series binary classification problems that simultaneously increase the accuracy and reliability of DNN outputs applied in a 5G UAV security dataset. These techniques use DNN outputs as input parameters and process them in different ways. Two methods use a well-known Machine Learning (ML) algorithm as a complement, and the other three use only confidence values that the DNN estimates. We compare seven different metrics, such as the Expected Calibration Error (ECE), Maximum Calibration Error (MCE), Mean Confidence (MC), Mean Accuracy (MA), Normalized Negative Log Likelihood (NLL), Brier Score Loss (BSL), and Reliability Score (RS) and the tradeoffs between them to evaluate the proposed hybrid algorithms. First, we show that the eXtreme Gradient Boosting (XGB) classifier might not be reliable for binary classification under the conditions this work presents. Second, we demonstrate that at least one of the potential methods can achieve better results than the classification in the DNN softmax layer. Finally, we show that the prospective methods may improve accuracy and reliability with better uncertainty calibration based on the assumption that the RS determines the difference between MC and MA metrics, and this difference should be zero to increase reliability. For example, Method 3 presents the best RS of 0.65 even when compared to the XGB classifier, which achieves RS of 7.22.