# ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification

arXiv: https://arxiv.org/abs/2202.06335 · HTML: https://mlsanity.com/paper/2202.06335/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 13 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [ERNIE: Enhanced Language Representation with Informative Entities](https://mlsanity.com/paper/1905.07129/) (2019)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://mlsanity.com/paper/1911.03343/) (2019)
- [Deep Packet: A Novel Approach For Encrypted Traffic Classification Using Deep Learning](https://mlsanity.com/paper/1709.02656/) (2017)
- [UER: An Open-Source Toolkit for Pre-training Models](https://mlsanity.com/paper/1909.05658/) (2019)
- [Deep Fingerprinting: Undermining Website Fingerprinting Defenses with Deep Learning](https://mlsanity.com/paper/1801.02265/) (2018)
- [k-fingerprinting: a Robust Scalable Website Fingerprinting Technique](https://mlsanity.com/paper/1509.00789/) (2015)
- [Deep Learning for Encrypted Traffic Classification: An Overview](https://mlsanity.com/paper/1810.07906/) (2018)
- [Robust Smartphone App Identification Via Encrypted Network Traffic Analysis](https://mlsanity.com/paper/1704.06099/) (2017)
- [PacketCGAN: Exploratory Study of Class Imbalance for Encrypted Traffic Classification Using CGAN](https://mlsanity.com/paper/1911.12046/) (2019)

## Built on by (most cited first, 30 of 66)

- [Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities](https://mlsanity.com/paper/2405.10825/) (2024)
- [NetLLM: Adapting Large Language Models for Networking](https://mlsanity.com/paper/2402.02338/) (2024)
- [NetGPT: Generative Pretrained Transformer for Network Traffic](https://mlsanity.com/paper/2304.09513/) (2023)
- [NetMamba: Efficient Network Traffic Classification via Pre-training Unidirectional Mamba](https://mlsanity.com/paper/2405.11449/) (2024)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [netFound: Principled Design for Network Foundation Models](https://mlsanity.com/paper/2310.17025/) (2023)
- [TrafficGPT: Breaking the Token Barrier for Efficient Long Traffic Analysis and Generation](https://mlsanity.com/paper/2403.05822/) (2024)
- [TFE-GNN: A Temporal Fusion Encoder Using Graph Neural Networks for Fine-grained Encrypted Traffic Classification](https://mlsanity.com/paper/2307.16713/) (2023)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Lens: A Knowledge-Guided Foundation Model for Network Traffic](https://mlsanity.com/paper/2402.03646/) (2024)
- [MoE2: Optimizing Collaborative Inference for Edge Large Language Models](https://mlsanity.com/paper/2501.09410/) (2025)
- [Brain-on-Switch: Towards Advanced Intelligent Network Data Plane via NN-Driven Traffic Analysis at Line-Speed](https://mlsanity.com/paper/2403.11090/) (2024)
- [DoLLM: How Large Language Models Understanding Network Flow Data to Detect Carpet Bombing DDoS](https://mlsanity.com/paper/2405.07638/) (2024)
- [Mapping the Landscape of Generative AI in Network Monitoring and Management](https://mlsanity.com/paper/2502.08576/) (2025)
- [Respond to Change with Constancy: Instruction-tuning with LLM for Non-I.I.D. Network Traffic Classification](https://mlsanity.com/paper/2505.20866/) (2025)
- [FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classification](https://mlsanity.com/paper/2508.19924/) (2025)
- [Decompose to Understand, Fuse to Detect: Frequency-Decoupled Anomaly Detection for Encrypted Network Traffic](https://mlsanity.com/paper/2605.02970/) (2026)
- [AutoML4ETC: Automated Neural Architecture Search for Real-World Encrypted Traffic Classification](https://mlsanity.com/paper/2308.02182/) (2023)
- [Towards a graph-based foundation model for network traffic analysis](https://mlsanity.com/paper/2409.08111/) (2024)
- [NetFlowGen: Leveraging Generative Pre-training for Network Traffic Dynamics](https://mlsanity.com/paper/2412.20635/) (2024)
- [Revolutionizing Encrypted Traffic Classification with MH-Net: A Multi-View Heterogeneous Graph Model](https://mlsanity.com/paper/2501.03279/) (2025)
- [Self-Supervised Transformer-based Contrastive Learning for Intrusion Detection Systems](https://mlsanity.com/paper/2505.08816/) (2025)
- [Demystifying Network Foundation Models](https://mlsanity.com/paper/2509.23089/) (2025)
- [NetBurst: Event-Centric Forecasting of Bursty, Intermittent Time Series](https://mlsanity.com/paper/2510.22397/) (2025)
- [PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization](https://mlsanity.com/paper/2603.13647/) (2026)
- [CENTILE: A Telemetry Foundation Model Evaluated by the Decisions It Drives](https://mlsanity.com/paper/2608.01725/) (2026)
- [Listen to Minority: Encrypted Traffic Classification for Class Imbalance with Contrastive Pre-Training](https://mlsanity.com/paper/2308.16453/) (2023)
- [One Train for Two Tasks: An Encrypted Traffic Classification Framework Using Supervised Contrastive Learning](https://mlsanity.com/paper/2402.07501/) (2024)
- [Generic Multi-modal Representation Learning for Network Traffic Analysis](https://mlsanity.com/paper/2405.02649/) (2024)
- [Efficiently and Effectively: A Two-stage Approach to Balance Plaintext and Encrypted Text for Traffic Classification](https://mlsanity.com/paper/2407.19687/) (2024)

## Abstract (arXiv)

Encrypted traffic classification requires discriminative and robust traffic representation captured from content-invisible and imbalanced traffic data for accurate classification, which is challenging but indispensable to achieve network security and network management. The major limitation of existing solutions is that they highly rely on the deep features, which are overly dependent on data size and hard to generalize on unseen data. How to leverage the open-domain unlabeled traffic data to learn representation with strong generalization ability remains a key challenge. In this paper,we propose a new traffic representation model called Encrypted Traffic Bidirectional Encoder Representations from Transformer (ET-BERT), which pre-trains deep contextualized datagram-level representation from large-scale unlabeled data. The pre-trained model can be fine-tuned on a small number of task-specific labeled data and achieves state-of-the-art performance across five encrypted traffic classification tasks, remarkably pushing the F1 of ISCX-Tor to 99.2% (4.4% absolute improvement), ISCX-VPN-Service to 98.9% (5.2% absolute improvement), Cross-Platform (Android) to 92.5% (5.4% absolute improvement), CSTNET-TLS 1.3 to 97.4% (10.0% absolute improvement). Notably, we provide explanation of the empirically powerful pre-training model by analyzing the randomness of ciphers. It gives us insights in understanding the boundary of classification ability over encrypted traffic. The code is available at: https://github.com/linwhitehat/ET-BERT.