# Demystifying Network Foundation Models

arXiv: https://arxiv.org/abs/2509.23089 · HTML: https://mlsanity.com/paper/2509.23089/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 27 of 27)

- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [Representation Degeneration Problem in Training Natural Language Generation Models](https://mlsanity.com/paper/1907.12009/) (2019)
- [Estimating the intrinsic dimension of datasets by a minimal neighborhood information](https://mlsanity.com/paper/1803.06992/) (2018)
- [Whitening Sentence Representations for Better Semantics and Faster Retrieval](https://mlsanity.com/paper/2103.15316/) (2021)
- [Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection](https://mlsanity.com/paper/1802.09089/) (2018)
- [The geometry of hidden representations of large transformer models](https://mlsanity.com/paper/2302.00294/) (2023)
- [Dos and Don'ts of Machine Learning in Computer Security](https://mlsanity.com/paper/2010.09470/) (2020)
- [Is Cosine-Similarity of Embeddings Really About Similarity?](https://mlsanity.com/paper/2403.05440/) (2024)
- [WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach](https://mlsanity.com/paper/2104.01767/) (2021)
- [ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification](https://mlsanity.com/paper/2202.06335/) (2022)
- [Towards Understanding Linear Word Analogies](https://mlsanity.com/paper/1810.04882/) (2018)
- [NetGPT: Generative Pretrained Transformer for Network Traffic](https://mlsanity.com/paper/2304.09513/) (2023)
- [NetDiffusion: Network Data Augmentation Through Protocol-Constrained Traffic Generation](https://mlsanity.com/paper/2310.08543/) (2023)
- [NetMamba: Efficient Network Traffic Classification via Pre-training Unidirectional Mamba](https://mlsanity.com/paper/2405.11449/) (2024)
- [Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity](https://mlsanity.com/paper/2306.00458/) (2023)
- [TrafficGPT: Breaking the Token Barrier for Efficient Long Traffic Analysis and Generation](https://mlsanity.com/paper/2403.05822/) (2024)
- [SoK: Decoding the Enigma of Encrypted Network Traffic Classifiers](https://mlsanity.com/paper/2503.20093/) (2025)
- [Taurus: A Data Plane Architecture for Per-Packet ML](https://mlsanity.com/paper/2002.08987/) (2020)
- [Token-Modification Adversarial Attacks for Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.00676/) (2021)
- [Measuring the Performance and Network Utilization of Popular Video Conferencing Applications](https://mlsanity.com/paper/2105.13478/) (2021)
- [CausalSim: A Causal Framework for Unbiased Trace-Driven Simulation](https://mlsanity.com/paper/2201.01811/) (2022)
- [NetBench: A Large-Scale and Comprehensive Network Traffic Benchmark Dataset for Foundation Models](https://mlsanity.com/paper/2403.10319/) (2024)
- [Evaluation of Machine Learning Classifiers for Zero-Day Intrusion Detection -- An Analysis on CIC-AWS-2018 dataset](https://mlsanity.com/paper/1905.03685/) (2019)
- [AC-DC: Adaptive Ensemble Classification for Network Traffic Identification](https://mlsanity.com/paper/2302.11718/) (2023)
- [Veritas: Answering Causal Queries from Video Streaming Traces](https://mlsanity.com/paper/2208.12596/) (2022)
- [Watching Stars in Pixels: The Interplay of Traffic Shaping and YouTube Streaming QoE over GEO Satellite Networks](https://mlsanity.com/paper/2309.00686/) (2023)

## Built on by (most cited first, 2 of 2)

- [NetBurst: Event-Centric Forecasting of Bursty, Intermittent Time Series](https://mlsanity.com/paper/2510.22397/) (2025)
- [Generalizable IoT Traffic Representations for Cross-Network Device Identification](https://mlsanity.com/paper/2601.19315/) (2026)

## Abstract (arXiv)

This work presents a systematic investigation into the latent knowledge encoded within Network Foundation Models (NFMs) that focuses on hidden representations analysis rather than pure downstream task performance. Different from existing efforts, we analyze the models through a three-part evaluation: Embedding Geometry Analysis to assess representation space utilization, Metric Alignment Assessment to measure correspondence with domain-expert features, and Causal Sensitivity Testing to evaluate robustness to protocol perturbations. Using five diverse network datasets spanning controlled and real-world environments, we evaluate four state-of-the-art NFMs, revealing that they all exhibit significant anisotropy, inconsistent feature sensitivity patterns, an inability to separate the high-level context, payload dependency, and other properties. Our work identifies numerous limitations across all models and demonstrates that addressing them can significantly improve model performance (by up to +0.35 F1 score without architectural changes).