# SecureBERT and LLAMA 2 Empowered Control Area Network Intrusion Detection and Classification

arXiv: https://arxiv.org/abs/2311.12074 · HTML: https://mlsanity.com/paper/2311.12074/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 19 Nov 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [SecureBERT: A Domain-Specific Language Model for Cybersecurity](https://mlsanity.com/paper/2204.02685/) (2022)
- [GIDS: GAN based Intrusion Detection System for In-Vehicle Network](https://mlsanity.com/paper/1907.07377/) (2019)
- [MTH-IDS: A Multi-Tiered Hybrid Intrusion Detection System for Internet of Vehicles](https://mlsanity.com/paper/2105.13289/) (2021)
- [CAN-BERT do it? Controller Area Network Intrusion Detection System based on BERT Language Model](https://mlsanity.com/paper/2210.09439/) (2022)
- [Viden: Attacker Identification on In-Vehicle Networks](https://mlsanity.com/paper/1708.08414/) (2017)
- [can-train-and-test: A Curated CAN Dataset for Automotive Intrusion Detection](https://mlsanity.com/paper/2308.04972/) (2023)
- [Data Mining with Big Data in Intrusion Detection Systems: A Systematic Literature Review](https://mlsanity.com/paper/2005.12267/) (2020)

## Built on by (most cited first, 2 of 2)

- [A Survey on Large Language Models for Communication, Network, and Service Management: Application Insights, Challenges, and Future Directions](https://mlsanity.com/paper/2412.19823/) (2024)
- [MistralBSM: Leveraging Mistral-7B for Vehicular Networks Misbehavior Detection](https://mlsanity.com/paper/2407.18462/) (2024)

## Abstract (arXiv)

Numerous studies have proved their effective strength in detecting Control Area Network (CAN) attacks. In the realm of understanding the human semantic space, transformer-based models have demonstrated remarkable effectiveness. Leveraging pre-trained transformers has become a common strategy in various language-related tasks, enabling these models to grasp human semantics more comprehensively. To delve into the adaptability evaluation on pre-trained models for CAN intrusion detection, we have developed two distinct models: CAN-SecureBERT and CAN-LLAMA2. Notably, our CAN-LLAMA2 model surpasses the state-of-the-art models by achieving an exceptional performance 0.999993 in terms of balanced accuracy, precision detection rate, F1 score, and a remarkably low false alarm rate of 3.10e-6. Impressively, the false alarm rate is 52 times smaller than that of the leading model, MTH-IDS (Multitiered Hybrid Intrusion Detection System). Our study underscores the promise of employing a Large Language Model as the foundational model, while incorporating adapters for other cybersecurity-related tasks and maintaining the model's inherent language-related capabilities.