# netFound: Principled Design for Network Foundation Models

arXiv: https://arxiv.org/abs/2310.17025 · HTML: https://mlsanity.com/paper/2310.17025/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 42 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection](https://mlsanity.com/paper/1802.09089/) (2018)
- [ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification](https://mlsanity.com/paper/2202.06335/) (2022)
- [Towards Understanding Linear Word Analogies](https://mlsanity.com/paper/1810.04882/) (2018)
- [Why (and How) Networks Should Run Themselves](https://mlsanity.com/paper/1710.11583/) (2017)
- [Knowledge-Defined Networking](https://mlsanity.com/paper/1606.06222/) (2016)

## Built on by (most cited first, 17 of 17)

- [NetLLM: Adapting Large Language Models for Networking](https://mlsanity.com/paper/2402.02338/) (2024)
- [LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning](https://mlsanity.com/paper/2401.16185/) (2024)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [TrafficGPT: Breaking the Token Barrier for Efficient Long Traffic Analysis and Generation](https://mlsanity.com/paper/2403.05822/) (2024)
- [Lens: A Knowledge-Guided Foundation Model for Network Traffic](https://mlsanity.com/paper/2402.03646/) (2024)
- [TrojFM: Resource-efficient Backdoor Attacks against Very Large Foundation Models](https://mlsanity.com/paper/2405.16783/) (2024)
- [Mapping the Landscape of Generative AI in Network Monitoring and Management](https://mlsanity.com/paper/2502.08576/) (2025)
- [Towards a graph-based foundation model for network traffic analysis](https://mlsanity.com/paper/2409.08111/) (2024)
- [Self-Supervised Transformer-based Contrastive Learning for Intrusion Detection Systems](https://mlsanity.com/paper/2505.08816/) (2025)
- [Making Logic a First-Class Citizen in Generative ML for Networking](https://mlsanity.com/paper/2506.23964/) (2025)
- [PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization](https://mlsanity.com/paper/2603.13647/) (2026)
- [FlowWise: Stateful Fast-Slow Model Serving for Streaming Traffic Intelligence](https://mlsanity.com/paper/2402.03694/) (2024)
- [LLMs' Suitability for Network Security: A Case Study of STRIDE Threat Modeling](https://mlsanity.com/paper/2505.04101/) (2025)
- [Zero-Trust Foundation Models: A New Paradigm for Secure and Collaborative Artificial Intelligence for Internet of Things](https://mlsanity.com/paper/2505.23792/) (2025)
- [PACC: Protocol-Aware Cross-Layer Compression for Compact Network Traffic Representation](https://mlsanity.com/paper/2602.08331/) (2026)
- [Algorithmic Data Minimization for Machine Learning over Internet-of-Things Data Streams](https://mlsanity.com/paper/2503.05675/) (2025)
- [Versatile yet Efficient Network Traffic Analysis: Offloading Network Foundation Model to SmartNIC](https://mlsanity.com/paper/2508.02001/) (2025)

## Abstract (arXiv)

Network foundation models promise reusable representations for diverse traffic analysis tasks, but recent diagnostic works have revealed fundamental problems: models exploit dataset shortcuts rather than learning genuine traffic patterns, produce collapsed embedding spaces, and fail to capture the exogenous network conditions that shape real-world behavior. We translate these diagnostic insights into four concrete design principles: protocol-aware tokenization, operational context embedding, burst-flow hierarchical attention, and privacy-by-construction input design, and build netFound, a network foundation model whose architecture is motivated by this failure analysis. We pretrain netFound on a billion-token-scale corpus over 5000 GPU hours, and demonstrate that it produces high-quality representations with lower anisotropy, significantly higher alignment with domain-expert features, and an F1 of 0.95 on exogenous context discrimination where existing state-of-the-art models score below 0.62, while preserving privacy by excluding payload and IP addresses. netFound demonstrates significant improvements in frozen-encoder evaluation, showing that pretrained embeddings themselves carry useful structure, and remains the top performer across all benchmarks in end-to-end fine-tuned settings. We release full open-source code, weights for three model sizes on HuggingFace, a containerized pipeline from raw PCAPs to downstream inference, and the full 4.2 billion flows pretraining dataset to facilitate reproducibility and further research.