# PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization

arXiv: https://arxiv.org/abs/2603.13647 · HTML: https://mlsanity.com/paper/2603.13647/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining](https://mlsanity.com/paper/2305.10429/) (2023)
- [Byte Latent Transformer: Patches Scale Better Than Tokens](https://mlsanity.com/paper/2412.09871/) (2024)
- [AnglE-optimized Text Embeddings](https://mlsanity.com/paper/2309.12871/) (2023)
- [NetLLM: Adapting Large Language Models for Networking](https://mlsanity.com/paper/2402.02338/) (2024)
- [ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification](https://mlsanity.com/paper/2202.06335/) (2022)
- [NetGPT: Generative Pretrained Transformer for Network Traffic](https://mlsanity.com/paper/2304.09513/) (2023)
- [NetDiffusion: Network Data Augmentation Through Protocol-Constrained Traffic Generation](https://mlsanity.com/paper/2310.08543/) (2023)
- [NetMamba: Efficient Network Traffic Classification via Pre-training Unidirectional Mamba](https://mlsanity.com/paper/2405.11449/) (2024)
- [netFound: Principled Design for Network Foundation Models](https://mlsanity.com/paper/2310.17025/) (2023)
- [TrafficGPT: Breaking the Token Barrier for Efficient Long Traffic Analysis and Generation](https://mlsanity.com/paper/2403.05822/) (2024)
- [TrafficLLM: Enhancing Large Language Models for Network Traffic Analysis with Generic Traffic Representation](https://mlsanity.com/paper/2504.04222/) (2025)
- [Lens: A Knowledge-Guided Foundation Model for Network Traffic](https://mlsanity.com/paper/2402.03646/) (2024)
- [LLMcap: Large Language Model for Unsupervised PCAP Failure Detection](https://mlsanity.com/paper/2407.06085/) (2024)

## Built on by (most cited first, 2 of 2)

- [Evidence-Grounded Ensemble Diagnosis of 802.11 Packet Captures: A Multi-Stage Pipeline with Deterministic Reliability Scoring](https://mlsanity.com/paper/2606.06871/) (2026)
- [Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models](https://mlsanity.com/paper/2606.20587/) (2026)

## Abstract (arXiv)

Foundation models succeed when they learn in the native structure of a modality, whether morphology-respecting tokens in language or pixels in vision. Wireless packet traces deserve the same treatment: meaning emerges from layered headers, typed fields, timing gaps, and cross-packet state machines, not flat strings. We present Plume (Protocol Language Understanding Model for Exchanges), a compact 140M-parameter foundation model for 802.11 traces that learns from structured PDML dissections. A protocol-aware tokenizer splits along the dissector field tree, emits gap tokens for timing, and normalizes identifiers, yielding 6.2x shorter sequences than BPE with higher per token information density. Trained on a curated corpus, Plume achieves 74-97% next-packet token accuracy across five real-world failure categories and AUROC >= 0.99 for zero-shot anomaly detection. On the same prediction task, frontier LLMs (Claude Opus 4.6, GPT-5.4) score comparably despite receiving identical protocol context, yet Plume does so with > 600x fewer parameters, fitting on a single GPU at effectively zero marginal cost vs. cloud API pricing, enabling on-prem, privacy-preserving root cause analysis.