# INSIGHT: A Survey of In-Network Systems for Intelligent, High-Efficiency AI and Topology Optimization

arXiv: https://arxiv.org/abs/2505.24269 · HTML: https://mlsanity.com/paper/2505.24269/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 46 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 15 of 15)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [N-BaIoT: Network-based Detection of IoT Botnet Attacks Using Deep Autoencoders](https://mlsanity.com/paper/1805.03409/) (2018)
- [Scaling Distributed Machine Learning with In-Network Aggregation](https://mlsanity.com/paper/1903.06701/) (2019)
- [An Overview of Neural Network Compression](https://mlsanity.com/paper/2006.03669/) (2020)
- [In-network Computation for Large-scale Federated Learning over Wireless Edge Networks](https://mlsanity.com/paper/2109.10903/) (2021)
- [Brain-on-Switch: Towards Advanced Intelligent Network Data Plane via NN-Driven Traffic Analysis at Line-Speed](https://mlsanity.com/paper/2403.11090/) (2024)
- [IIsy: Practical In-Network Classification](https://mlsanity.com/paper/2205.08243/) (2022)
- [Quark: Implementing Convolutional Neural Networks Entirely on Programmable Data Plane](https://mlsanity.com/paper/2501.15100/) (2025)
- [NetFC: enabling accurate floating-point arithmetic on programmable switches](https://mlsanity.com/paper/2106.05467/) (2021)
- [A Review of the In-Network Computing and Its Role in the Edge-Cloud Continuum](https://mlsanity.com/paper/2312.00303/) (2023)

## Abstract (arXiv)

In-network computation represents a transformative approach to addressing the escalating demands of Artificial Intelligence (AI) workloads on network infrastructure. By leveraging the processing capabilities of network devices such as switches, routers, and Network Interface Cards (NICs), this paradigm enables AI computations to be performed directly within the network fabric, significantly reducing latency, enhancing throughput, and optimizing resource utilization. This paper provides a comprehensive analysis of optimizing in-network computation for AI, exploring the evolution of programmable network architectures, such as Software-Defined Networking (SDN) and Programmable Data Planes (PDPs), and their convergence with AI. It examines methodologies for mapping AI models onto resource-constrained network devices, addressing challenges like limited memory and computational capabilities through efficient algorithm design and model compression techniques. The paper also highlights advancements in distributed learning, particularly in-network aggregation, and the potential of federated learning to enhance privacy and scalability. Frameworks like Planter and Quark are discussed for simplifying development, alongside key applications such as intelligent network monitoring, intrusion detection, traffic management, and Edge AI. Future research directions, including runtime programmability, standardized benchmarks, and new applications paradigms, are proposed to advance this rapidly evolving field. This survey underscores the potential of in-network AI to create intelligent, efficient, and responsive networks capable of meeting the demands of next-generation AI applications.