# Running Neural Networks on the NIC

arXiv: https://arxiv.org/abs/2009.02353 · HTML: https://mlsanity.com/paper/2009.02353/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2020.

## Built on by (most cited first, 6 of 6)

- [FENXI: Deep-learning Traffic Analytics at the Edge](https://mlsanity.com/paper/2105.11738/) (2021)
- [Brain-on-Switch: Towards Advanced Intelligent Network Data Plane via NN-Driven Traffic Analysis at Line-Speed](https://mlsanity.com/paper/2403.11090/) (2024)
- [IIsy: Practical In-Network Classification](https://mlsanity.com/paper/2205.08243/) (2022)
- [Landing AI on Networks: An equipment vendor viewpoint on Autonomous Driving Networks](https://mlsanity.com/paper/2205.08347/) (2022)
- [KML: Using Machine Learning to Improve Storage Systems](https://mlsanity.com/paper/2111.11554/) (2021)
- [Automating In-Network Machine Learning](https://mlsanity.com/paper/2205.08824/) (2022)

## Abstract (arXiv)

In this paper we show that the data plane of commodity programmable (Network Interface Cards) NICs can run neural network inference tasks required by packet monitoring applications, with low overhead. This is particularly important as the data transfer costs to the host system and dedicated machine learning accelerators, e.g., GPUs, can be more expensive than the processing task itself. We design and implement our system -- N3IC -- on two different NICs and we show that it can greatly benefit three different network monitoring use cases that require machine learning inference as first-class-primitive. N3IC can perform inference for millions of network flows per second, while forwarding traffic at 40Gb/s. Compared to an equivalent solution implemented on a general purpose CPU, N3IC can provide 100x lower processing latency, with 1.5x increase in throughput.