# NNStreamer: Efficient and Agile Development of On-Device AI Systems

arXiv: https://arxiv.org/abs/2101.06371 · HTML: https://mlsanity.com/paper/2101.06371/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 53 references that are in the map.
- First posted on arXiv 16 Jan 2021.

## Builds on (most cited first, 8 of 8)

- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Instance-aware Semantic Segmentation via Multi-task Network Cascades](https://mlsanity.com/paper/1512.04412/) (2015)
- [AI Benchmark: Running Deep Neural Networks on Android Smartphones](https://mlsanity.com/paper/1810.01109/) (2018)
- [TVM: An Automated End-to-End Optimizing Compiler for Deep Learning](https://mlsanity.com/paper/1802.04799/) (2018)
- [Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning](https://mlsanity.com/paper/1801.08058/) (2018)

## Built on by (most cited first, 2 of 2)

- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [Toward Among-Device AI from On-Device AI with Stream Pipelines](https://mlsanity.com/paper/2201.06026/) (2022)

## Abstract (arXiv)

We propose NNStreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to deep neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI. It is to process neural networks on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data transmission costs, and operational costs signify the need for on-device AI, especially if we deploy a massive number of devices. NNStreamer efficiently handles neural networks with complex data stream pipelines on devices, significantly improving the overall performance with minimal efforts. Besides, NNStreamer simplifies implementations and allows reusing off-the-shelf media filters directly, which reduces developmental costs significantly. We are already deploying NNStreamer for a wide range of products and platforms, including the Galaxy series and various consumer electronic devices. The experimental results suggest a reduction in developmental costs and enhanced performance of pipeline architectures and NNStreamer. It is an open-source project incubated by Linux Foundation AI, available to the public and applicable to various hardware and software platforms.