# Efficient Federated Intrusion Detection in 5G ecosystem using optimized BERT-based model

arXiv: https://arxiv.org/abs/2409.19390 · HTML: https://mlsanity.com/paper/2409.19390/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 28 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning](https://mlsanity.com/paper/2402.06954/) (2024)
- [Neural Machine Translation with Byte-Level Subwords](https://mlsanity.com/paper/1909.03341/) (2019)
- [CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models](https://mlsanity.com/paper/2404.13161/) (2024)
- [Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices](https://mlsanity.com/paper/2306.14263/) (2023)
- [FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition](https://mlsanity.com/paper/2404.18848/) (2024)
- [Federated Learning for Intrusion Detection System: Concepts, Challenges and Future Directions](https://mlsanity.com/paper/2106.09527/) (2021)
- [DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model](https://mlsanity.com/paper/2404.05182/) (2024)
- [Federated Learning for Zero-Day Attack Detection in 5G and Beyond V2X Networks](https://mlsanity.com/paper/2407.03070/) (2024)
- [Security in 5G Networks -- How 5G networks help Mitigate Location Tracking Vulnerability](https://mlsanity.com/paper/2312.16200/) (2023)

## Built on by (most cited first, 8 of 8)

- [When IoT Meet LLMs: Applications and Challenges](https://mlsanity.com/paper/2411.17722/) (2024)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [LLM-Based Threat Detection and Prevention Framework for IoT Ecosystems](https://mlsanity.com/paper/2505.00240/) (2025)
- [A Unified Framework for Context-Aware IoT Management and State-of-the-Art IoT Traffic Anomaly Detection](https://mlsanity.com/paper/2412.19830/) (2024)
- [Integrated LLM-Based Intrusion Detection with Secure Slicing xApp for Securing O-RAN-Enabled Wireless Network Deployments](https://mlsanity.com/paper/2504.00341/) (2025)
- [AI-Native Closed-Loop Security for 6G-Enabled Cyber-Physical Systems: From Edge Detection to Network-Wide Mitigation](https://mlsanity.com/paper/2606.08173/) (2026)
- [Zero-Trust Agentic Federated Learning for Secure IIoT Defense Systems](https://mlsanity.com/paper/2512.23809/) (2025)
- [Attribution-Driven Explainable Intrusion Detection with Encoder-Based Large Language Models](https://mlsanity.com/paper/2604.06266/) (2026)

## Abstract (arXiv)

The fifth-generation (5G) offers advanced services, supporting applications such as intelligent transportation, connected healthcare, and smart cities within the Internet of Things (IoT). However, these advancements introduce significant security challenges, with increasingly sophisticated cyber-attacks. This paper proposes a robust intrusion detection system (IDS) using federated learning and large language models (LLMs). The core of our IDS is based on BERT, a transformer model adapted to identify malicious network flows. We modified this transformer to optimize performance on edge devices with limited resources. Experiments were conducted in both centralized and federated learning contexts. In the centralized setup, the model achieved an inference accuracy of 97.79%. In a federated learning context, the model was trained across multiple devices using both IID (Independent and Identically Distributed) and non-IID data, based on various scenarios, ensuring data privacy and compliance with regulations. We also leveraged linear quantization to compress the model for deployment on edge devices. This reduction resulted in a slight decrease of 0.02% in accuracy for a model size reduction of 28.74%. The results underscore the viability of LLMs for deployment in IoT ecosystems, highlighting their ability to operate on devices with constrained computational and storage resources.