# Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey

arXiv: https://arxiv.org/abs/2408.07583 · HTML: https://mlsanity.com/paper/2408.07583/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 141 references that are in the map.
- First posted on arXiv 14 Aug 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [Auto-Keras: An Efficient Neural Architecture Search System](https://mlsanity.com/paper/1806.10282/) (2018)
- [LogBERT: Log Anomaly Detection via BERT](https://mlsanity.com/paper/2103.04475/) (2021)
- [Large Generative AI Models for Telecom: The Next Big Thing?](https://mlsanity.com/paper/2306.10249/) (2023)
- [Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey](https://mlsanity.com/paper/2403.01255/) (2024)
- [Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices](https://mlsanity.com/paper/2306.14263/) (2023)
- [LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model](https://mlsanity.com/paper/2111.09564/) (2021)
- [Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization](https://mlsanity.com/paper/2304.14535/) (2023)
- [FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2304.14746/) (2023)
- [Deep transfer learning for intrusion detection in industrial control networks: A comprehensive review](https://mlsanity.com/paper/2304.10550/) (2023)
- [Can Large Language Models Provide Security & Privacy Advice? Measuring the Ability of LLMs to Refute Misconceptions](https://mlsanity.com/paper/2310.02431/) (2023)
- [AI in Thyroid Cancer Diagnosis: Techniques, Trends, and Future Directions](https://mlsanity.com/paper/2308.13592/) (2023)
- [CAN-BERT do it? Controller Area Network Intrusion Detection System based on BERT Language Model](https://mlsanity.com/paper/2210.09439/) (2022)
- [Deep Learning for Steganalysis of Diverse Data Types: A review of methods, taxonomy, challenges and future directions](https://mlsanity.com/paper/2308.04522/) (2023)
- [AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generation](https://mlsanity.com/paper/2310.02655/) (2023)
- [DeepSQLi: Deep Semantic Learning for Testing SQL Injection](https://mlsanity.com/paper/2005.11728/) (2020)
- [Enhancing IoT Security with CNN and LSTM-Based Intrusion Detection Systems](https://mlsanity.com/paper/2405.18624/) (2024)
- [On Generalisability of Machine Learning-based Network Intrusion Detection Systems](https://mlsanity.com/paper/2205.04112/) (2022)
- [Deep Reinforcement Learning for Intrusion Detection in IoT: A Survey](https://mlsanity.com/paper/2405.20038/) (2024)
- [Collaborative Feature Maps of Networks and Hosts for AI-driven Intrusion Detection](https://mlsanity.com/paper/2208.05085/) (2022)
- [On Improving Deep Learning Trace Analysis with System Call Arguments](https://mlsanity.com/paper/2103.06915/) (2021)
- [Flow-based Network Intrusion Detection Based on BERT Masked Language Model](https://mlsanity.com/paper/2306.04920/) (2023)
- [A Method for Network Intrusion Detection Using Flow Sequence and BERT Framework](https://mlsanity.com/paper/2310.17127/) (2023)

## Built on by (most cited first, 21 of 21)

- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Automatic Speech Recognition with BERT and CTC Transformers: A Review](https://mlsanity.com/paper/2410.09456/) (2024)
- [Adaptive Cyber-Attack Detection in IIoT Using Attention-Based LSTM-CNN Models](https://mlsanity.com/paper/2501.13962/) (2025)
- [Advanced Deep Learning and Large Language Models: Comprehensive Insights for Cancer Detection](https://mlsanity.com/paper/2504.13186/) (2025)
- [Recent Advances in Transformer and Large Language Models for UAV Applications](https://mlsanity.com/paper/2508.11834/) (2025)
- [Deep Learning Techniques for Hand Vein Biometrics: A Comprehensive Review](https://mlsanity.com/paper/2409.07128/) (2024)
- [Advances in Artificial Intelligence: A Review for the Creative Industries](https://mlsanity.com/paper/2501.02725/) (2025)
- [Interpretable Anomaly-Based DDoS Detection in AI-RAN with XAI and LLMs](https://mlsanity.com/paper/2507.21193/) (2025)
- [A Robust Cross-Domain IDS using BiGRU-LSTM-Attention for Medical and Industrial IoT Security](https://mlsanity.com/paper/2508.12470/) (2025)
- [Multi-Stage Retrieval for Operational Technology Cybersecurity Compliance Using Large Language Models: A Railway Casestudy](https://mlsanity.com/paper/2504.14044/) (2025)
- [Improving Pretrained YAMNet for Enhanced Speech Command Detection via Transfer Learning](https://mlsanity.com/paper/2504.19030/) (2025)
- [Enhancing Cochlear Implant Signal Coding with Scaled Dot-Product Attention](https://mlsanity.com/paper/2504.19046/) (2025)
- [Exploring Kolmogorov-Arnold Network Expansions in Vision Transformers for Mitigating Catastrophic Forgetting in Continual Learning](https://mlsanity.com/paper/2507.04020/) (2025)
- [Enhancing GraphQL Security by Detecting Malicious Queries Using Large Language Models, Sentence Transformers, and Convolutional Neural Networks](https://mlsanity.com/paper/2508.11711/) (2025)
- [Geometric Capacity of Transformers: A Tropical Geometry Perspective](https://mlsanity.com/paper/2604.14727/) (2026)
- [Transfer Learning-Based Deep Residual Learning for Speech Recognition in Clean and Noisy Environments](https://mlsanity.com/paper/2505.01632/) (2025)
- [Spatio-Temporal Attention Graph Neural Network: Explaining Causalities With Attention](https://mlsanity.com/paper/2603.10676/) (2026)
- [Hybrid ResNet-1D-BiGRU with Multi-Head Attention for Cyberattack Detection in Industrial IoT Environments](https://mlsanity.com/paper/2604.06481/) (2026)
- [GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attacks](https://mlsanity.com/paper/2606.05844/) (2026)
- [Do Transformers Actually Help Intrusion Detection? A Temporal Sequence Evaluation on CIC-IDS2017](https://mlsanity.com/paper/2606.11098/) (2026)
- [Multi-Level Distributional Entropy from Flow Summary Statistics for Explainable Network Intrusion Detection](https://mlsanity.com/paper/2606.29797/) (2026)

## Abstract (arXiv)

With significant advancements in Transformers LLMs, NLP has extended its reach into many research fields due to its enhanced capabilities in text generation and user interaction. One field benefiting greatly from these advancements is cybersecurity. In cybersecurity, many parameters that need to be protected and exchanged between senders and receivers are in the form of text and tabular data, making NLP a valuable tool in enhancing the security measures of communication protocols. This survey paper provides a comprehensive analysis of the utilization of Transformers and LLMs in cyber-threat detection systems. The methodology of paper selection and bibliometric analysis is outlined to establish a rigorous framework for evaluating existing research. The fundamentals of Transformers are discussed, including background information on various cyber-attacks and datasets commonly used in this field. The survey explores the application of Transformers in IDSs, focusing on different architectures such as Attention-based models, LLMs like BERT and GPT, CNN/LSTM-Transformer hybrids, emerging approaches like ViTs, among others. Furthermore, it explores the diverse environments and applications where Transformers and LLMs-based IDS have been implemented, including computer networks, IoT devices, critical infrastructure protection, cloud computing, SDN, as well as in autonomous vehicles. The paper also addresses research challenges and future directions in this area, identifying key issues such as interpretability, scalability, and adaptability to evolving threats, and more. Finally, the conclusion summarizes the findings and highlights the significance of Transformers and LLMs in enhancing cyber-threat detection capabilities, while also outlining potential avenues for further research and development.