# A new hope for network model generalization

arXiv: https://arxiv.org/abs/2207.05843 · HTML: https://mlsanity.com/paper/2207.05843/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 43 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Better plain ViT baselines for ImageNet-1k](https://mlsanity.com/paper/2205.01580/) (2022)
- [Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches](https://mlsanity.com/paper/1904.01172/) (2019)
- [Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks](https://mlsanity.com/paper/1712.00162/) (2017)
- [Automatic Curriculum Generation for Learning Adaptation in Networking](https://mlsanity.com/paper/2202.05940/) (2022)
- [Factorization tricks for LSTM networks](https://mlsanity.com/paper/1703.10722/) (2017)
- [BERT-CoQAC: BERT-based Conversational Question Answering in Context](https://mlsanity.com/paper/2104.11394/) (2021)
- [Real-world Video Adaptation with Reinforcement Learning](https://mlsanity.com/paper/2008.12858/) (2020)
- [Homa: A Receiver-Driven Low-Latency Transport Protocol Using Network Priorities (Complete Version)](https://mlsanity.com/paper/1803.09615/) (2018)

## Built on by (most cited first, 11 of 11)

- [RACONTEUR: A Knowledgeable, Insightful, and Portable LLM-Powered Shell Command Explainer](https://mlsanity.com/paper/2409.02074/) (2024)
- [LogPr'ecis: Unleashing Language Models for Automated Malicious Log Analysis](https://mlsanity.com/paper/2307.08309/) (2023)
- [NetFlowGen: Leveraging Generative Pre-training for Network Traffic Dynamics](https://mlsanity.com/paper/2412.20635/) (2024)
- [Self-Supervised Transformer-based Contrastive Learning for Intrusion Detection Systems](https://mlsanity.com/paper/2505.08816/) (2025)
- [An LLM-based Agentic Framework for Accessible Network Control](https://mlsanity.com/paper/2509.20600/) (2025)
- [Modelling Concurrent RTP Flows for End-to-end Predictions of QoS in Real Time Communications](https://mlsanity.com/paper/2410.15846/) (2024)
- [Application of Tabular Transformer Architectures for Operating System Fingerprinting](https://mlsanity.com/paper/2502.09084/) (2025)
- [m4: A Learned Flow-level Network Simulator](https://mlsanity.com/paper/2503.01770/) (2025)
- [Which Deep Learner? A Systematic Evaluation of Advanced Deep Forecasting Models Accuracy and Efficiency for Network Traffic Prediction](https://mlsanity.com/paper/2601.02694/) (2026)
- [SwiftQueue: Optimizing Low-Latency Applications with Swift Packet Queuing](https://mlsanity.com/paper/2410.06112/) (2024)
- [From Simulation to Deep Learning: Survey on Network Performance Modeling Approaches](https://mlsanity.com/paper/2603.28394/) (2026)

## Abstract (arXiv)

Generalizing machine learning (ML) models for network traffic dynamics tends to be considered a lost cause. Hence for every new task, we design new models and train them on model-specific datasets closely mimicking the deployment environments. Yet, an ML architecture called_Transformer_ has enabled previously unimaginable generalization in other domains. Nowadays, one can download a model pre-trained on massive datasets and only fine-tune it for a specific task and context with comparatively little time and data. These fine-tuned models are now state-of-the-art for many benchmarks. We believe this progress could translate to networking and propose a Network Traffic Transformer (NTT), a transformer adapted to learn network dynamics from packet traces. Our initial results are promising: NTT seems able to generalize to new prediction tasks and environments. This study suggests there is still hope for generalization, though it calls for a lot of future research.