# Evaluating False Alarm and Missing Attacks in CAN IDS

arXiv: https://arxiv.org/abs/2602.02781 · HTML: https://mlsanity.com/paper/2602.02781/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 13 of 13)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks](https://mlsanity.com/paper/1905.00441/) (2019)
- [A Comprehensive Guide to CAN IDS Data & Introduction of the ROAD Dataset](https://mlsanity.com/paper/2012.14600/) (2020)
- [CANShield: Deep Learning-Based Intrusion Detection Framework for Controller Area Networks at the Signal-Level](https://mlsanity.com/paper/2205.01306/) (2022)
- [Time-Based CAN Intrusion Detection Benchmark](https://mlsanity.com/paper/2101.05781/) (2021)
- [Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN](https://mlsanity.com/paper/2406.13778/) (2024)
- [On the Resilience of Machine Learning-Based IDS for Automotive Networks](https://mlsanity.com/paper/2306.14782/) (2023)
- [Detecting Masquerade Attacks in Controller Area Networks Using Graph Machine Learning](https://mlsanity.com/paper/2408.05427/) (2024)

## Abstract (arXiv)

Modern vehicles rely on electronic control units (ECUs) interconnected through the Controller Area Network (CAN), making in-vehicle communication a critical security concern. Machine learning (ML)-based intrusion detection systems (IDS) are increasingly deployed to protect CAN traffic, yet their robustness against adversarial manipulation remains largely unexplored. We present a systematic adversarial evaluation of CAN IDS using the ROAD dataset, comparing four shallow learning models with a deep neural network-based detector. Using protocol-compliant, payload-level perturbations generated via FGSM, BIM and PGD, we evaluate adversarial effects on both benign and malicious CAN frames. While all models achieve strong baseline performance under benign conditions, adversarial perturbations reveal substantial vulnerabilities. Although shallow and deep models are robust to false-alarm induction, with the deep neural network (DNN) performing best on benign traffic, all architectures suffer significant increases in missed attacks. Notably, under gradient-based attacks, the shallow model extra trees (ET) demonstrates improved robustness to missed-attack induction compared to the other models. Our results demonstrate that adversarial manipulation can simultaneously trigger false alarms and evade detection, underscoring the need for adversarial robustness evaluation in safety-critical automotive IDS.