mlsanityData to 7 Oct 2026

Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey

arXiv 2408.07583 ↗First posted 14 Aug 2024cs.CR cs.AI cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

With significant advancements in Transformers LLMs, NLP has extended its reach into many research fields due to its enhanced capabilities in text generation and user interaction. One field benefiting greatly from these advancements is cybersecurity. In cybersecurity, many parameters that need to be protected and exchanged between senders and receivers are in the form of text and tabular data, making NLP a valuable tool in enhancing the security measures of communication protocols. This survey paper provides a comprehensive analysis of the utilization of Transformers and LLMs in cyber-threat detection systems. The methodology of paper selection and bibliometric analysis is outlined to establish a rigorous framework for evaluating existing research. The fundamentals of Transformers are discussed, including background information on various cyber-attacks and datasets commonly used in this field. The survey explores the application of Transformers in IDSs, focusing on different architectures such as Attention-based models, LLMs like BERT and GPT, CNN/LSTM-Transformer hybrids, emerging approaches like ViTs, among others. Furthermore, it explores the diverse environments and applications where Transformers and LLMs-based IDS have been implemented, including computer networks, IoT devices, critical infrastructure protection, cloud computing, SDN, as well as in autonomous vehicles. The paper also addresses research challenges and future directions in this area, identifying key issues such as interpretability, scalability, and adaptability to evolving threats, and more. Finally, the conclusion summarizes the findings and highlights the significance of Transformers and LLMs in enhancing cyber-threat detection capabilities, while also outlining potential avenues for further research and development.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 25 of its 141 references that are in the map. First posted on arXiv 14 Aug 2024.

Builds on

25 of 141 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Knowledge Distillation: A Surveycited by 1,578
  3. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  4. 2018Auto-Keras: An Efficient Neural Architecture Search Systemcited by 169
  5. 2021LogBERT: Log Anomaly Detection via BERTcited by 114
  6. 2023Large Generative AI Models for Telecom: The Next Big Thing?cited by 52
  7. 2024Automatic Speech Recognition using Advanced Deep Learning Approaches: A surveycited by 50
  8. 2023Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devicescited by 38
  9. 2021LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Modelcited by 30
  10. 2023Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalizationcited by 28
  11. 2023FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systemscited by 24
  12. 2023Deep transfer learning for intrusion detection in industrial control networks: A comprehensive reviewcited by 21
13 more
  1. 2023Can Large Language Models Provide Security & Privacy Advice? Measuring the Ability of LLMs to Refute Misconceptionscited by 20
  2. 2023AI in Thyroid Cancer Diagnosis: Techniques, Trends, and Future Directionscited by 18
  3. 2022CAN-BERT do it? Controller Area Network Intrusion Detection System based on BERT Language Modelcited by 17
  4. 2023Deep Learning for Steganalysis of Diverse Data Types: A review of methods, taxonomy, challenges and future directionscited by 16
  5. 2023AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generationcited by 15
  6. 2020DeepSQLi: Deep Semantic Learning for Testing SQL Injectioncited by 11
  7. 2024Enhancing IoT Security with CNN and LSTM-Based Intrusion Detection Systemscited by 9
  8. 2022On Generalisability of Machine Learning-based Network Intrusion Detection Systemscited by 8
  9. 2024Deep Reinforcement Learning for Intrusion Detection in IoT: A Surveycited by 7
  10. 2022Collaborative Feature Maps of Networks and Hosts for AI-driven Intrusion Detectioncited by 4
  11. 2021On Improving Deep Learning Trace Analysis with System Call Argumentscited by 3
  12. 2023Flow-based Network Intrusion Detection Based on BERT Masked Language Modelcited by 2
  13. 2023A Method for Network Intrusion Detection Using Flow Sequence and BERT Frameworkcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 13; 2026: 6.

21 papers.

Sort:

By area:

  1. 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
  2. 2024Automatic Speech Recognition with BERT and CTC Transformers: A Reviewcited by 8
  3. 2025Adaptive Cyber-Attack Detection in IIoT Using Attention-Based LSTM-CNN Modelscited by 4
  4. 2025Advanced Deep Learning and Large Language Models: Comprehensive Insights for Cancer Detectioncited by 4
  5. 2025Recent Advances in Transformer and Large Language Models for UAV Applicationscited by 3
  6. 2024Deep Learning Techniques for Hand Vein Biometrics: A Comprehensive Reviewcited by 2
  7. 2025Advances in Artificial Intelligence: A Review for the Creative Industriescited by 2
  8. 2025A Robust Cross-Domain IDS using BiGRU-LSTM-Attention for Medical and Industrial IoT Securitycited by 2
  9. 2025Multi-Stage Retrieval for Operational Technology Cybersecurity Compliance Using Large Language Models: A Railway Casestudycited by 1
  10. 2025Improving Pretrained YAMNet for Enhanced Speech Command Detection via Transfer Learningcited by 1
  11. 2025Enhancing Cochlear Implant Signal Coding with Scaled Dot-Product Attentioncited by 1
  12. 2025Interpretable Anomaly-Based DDoS Detection in AI-RAN with XAI and LLMscited by 1
9 more
  1. 2025Enhancing GraphQL Security by Detecting Malicious Queries Using Large Language Models, Sentence Transformers, and Convolutional Neural Networkscited by 1
  2. 2026Geometric Capacity of Transformers: A Tropical Geometry Perspectivecited by 1
  3. 2025Transfer Learning-Based Deep Residual Learning for Speech Recognition in Clean and Noisy Environmentscited by 0
  4. 2025Exploring Kolmogorov-Arnold Network Expansions in Vision Transformers for Mitigating Catastrophic Forgetting in Continual Learningcited by 0
  5. 2026Spatio-Temporal Attention Graph Neural Network: Explaining Causalities With Attentioncited by 0
  6. 2026Hybrid ResNet-1D-BiGRU with Multi-Head Attention for Cyberattack Detection in Industrial IoT Environmentscited by 0
  7. 2026GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attackscited by 0
  8. 2026Do Transformers Actually Help Intrusion Detection? A Temporal Sequence Evaluation on CIC-IDS2017cited by 0
  9. 2026Multi-Level Distributional Entropy from Flow Summary Statistics for Explainable Network Intrusion Detectioncited by 0