# Protocol-Aware Tokenization and Architecture Co-Design for Wireless Packet Foundation Models

arXiv: https://arxiv.org/abs/2606.20587 · HTML: https://mlsanity.com/paper/2606.20587/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 14 May 2026.

## Builds on (most cited first, 11 of 11)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Byte Latent Transformer: Patches Scale Better Than Tokens](https://mlsanity.com/paper/2412.09871/) (2024)
- [NetGPT: Generative Pretrained Transformer for Network Traffic](https://mlsanity.com/paper/2304.09513/) (2023)
- [NetMamba: Efficient Network Traffic Classification via Pre-training Unidirectional Mamba](https://mlsanity.com/paper/2405.11449/) (2024)
- [TrafficGPT: Breaking the Token Barrier for Efficient Long Traffic Analysis and Generation](https://mlsanity.com/paper/2403.05822/) (2024)
- [LLMcap: Large Language Model for Unsupervised PCAP Failure Detection](https://mlsanity.com/paper/2407.06085/) (2024)
- [NetSSM: Multi-Flow and State-Aware Network Trace Generation using State Space Models](https://mlsanity.com/paper/2503.22663/) (2025)
- [PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization](https://mlsanity.com/paper/2603.13647/) (2026)

## Abstract (arXiv)

What matters more for building foundation models for wireless packet traces: the tokenizer or the architecture or both? To answer this question, we build on PLUME Anonymous [2026], which introduced protocol-aware tokenization for 802.11 traces; we scale model depth and transfer the same tokenizer to a fundamentally different architecture family. A deeper GPT (PLUME-DEEP, 24 layers) reaches 98.2% top-1 accuracy, gaining 32 points over the original 12-layer design, while a Mamba-2 state-space variant (PLUME-MAMBA) achieves 96.1% with 1.7x higher throughput and 2x longer context. The key insight emerges from a controlled 2x2 comparison across tokenizers and architectures: changing the tokenizer swings accuracy by 32 points; changing the architecture moves it by only 2. Protocol-aware tokenization is the primary performance lever, and the backbone becomes a deployment knob trading accuracy for speed.