mlsanityData to 14 Oct 2026

MNN: A Universal and Efficient Inference Engine

arXiv 2002.12418 ↗First posted 27 Feb 2020cs.CV cs.DC cs.LGcited by 65

Not on the map. Why?

Abstract (arXiv)

Deploying deep learning models on mobile devices draws more and more attention recently. However, designing an efficient inference engine on devices is under the great challenges of model compatibility, device diversity, and resource limitation. To deal with these challenges, we propose Mobile Neural Network (MNN), a universal and efficient inference engine tailored to mobile applications. In this paper, the contributions of MNN include: (1) presenting a mechanism called pre-inference that manages to conduct runtime optimization; (2)deliveringthorough kernel optimization on operators to achieve optimal computation performance; (3) introducing backend abstraction module which enables hybrid scheduling and keeps the engine lightweight. Extensive benchmark experiments demonstrate that MNN performs favorably against other popular lightweight deep learning frameworks. MNN is available to public at: https://github.com/alibaba/MNN.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 11; 2023: 8; 2024: 14; 2025: 10; 2026: 12.

65 papers.

Sort:

By area:

  1. 2022On-Device Training Under 256KB Memorycited by 160
  2. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
  3. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  4. 2024Fast On-device LLM Inference with NPUscited by 62
  5. 2021DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusioncited by 57
  6. 2024Tiny Machine Learning: Progress and Futurescited by 40
  7. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  8. 2022Learning Torque Control for Quadrupedal Locomotioncited by 29
  9. 2022ParC-Net: Position Aware Circular Convolution with Merits from ConvNets and Transformercited by 28
  10. 2023TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Trainingcited by 27
  11. 2024Galaxy: A Resource-Efficient Collaborative Edge AI System for In-situ Transformer Inferencecited by 25
  12. 2022On-device Training: A First Overview on Existing Systemscited by 19
38 more
  1. 2023Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devicescited by 17
  2. 2024Transformer-Lite: High-efficiency Deployment of Large Language Models on Mobile Phone GPUscited by 17
  3. 2021SparseDNN: Fast Sparse Deep Learning Inference on CPUscited by 13
  4. 2022RepGhost: A Hardware-Efficient Ghost Module via Re-parameterizationcited by 13
  5. 2024Asteroid: Resource-Efficient Hybrid Pipeline Parallelism for Collaborative DNN Training on Heterogeneous Edge Devicescited by 13
  6. 2024SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobilecited by 12
  7. 2025TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented Reality via 3D Gaussian Splattingcited by 11
  8. 2021GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsitycited by 10
  9. 2023CP3: Channel Pruning Plug-in for Point-based Networkscited by 10
  10. 2023FedTherapist: Mental Health Monitoring with User-Generated Linguistic Expressions on Smartphones via Federated Learningcited by 10
  11. 2021FastICARL: Fast Incremental Classifier and Representation Learning with Efficient Budget Allocation in Audio Sensing Applicationscited by 9
  12. 2023Enabling Resource-efficient AIoT System with Cross-level Optimization: A surveycited by 8
  13. 2025Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selectioncited by 8
  14. 2025MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devicescited by 8
  15. 2022On-Device Learning with Cloud-Coordinated Data Augmentation for Extreme Model Personalization in Recommender Systemscited by 6
  16. 2022Boosting DNN Cold Inference on Edge Devicescited by 6
  17. 2024TwinLiteNet+: An Enhanced Multi-Task Segmentation Model for Autonomous Drivingcited by 5
  18. 2025Dissecting the Impact of Mobile DVFS Governors on LLM Inference Performance and Energy Efficiencycited by 5
  19. 2026ExecuTorch -- A Unified PyTorch Solution to Run AI Models On-Devicecited by 5
  20. 2026Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agentscited by 5
  21. 2021DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensingcited by 4
  22. 2022Enabling On-Device Smartphone GPU based Training: Lessons Learnedcited by 4
  23. 2023Confidant: Customizing Transformer-based LLMs via Collaborative Edge Trainingcited by 3
  24. 2024Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Trainingcited by 3
  25. 2024A Scene-aware Models Adaptation Scheme for Cross-scene Online Inference on Mobile Devicescited by 3
  26. 2024Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligencecited by 3
  27. 2024Elastic On-Device LLM Servicecited by 3
  28. 2025Scaling On-Device GPU Inference for Large Generative Modelscited by 3
  29. 2025Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inferencecited by 3
  30. 2021SoftNeuro: Fast Deep Inference using Multi-platform Optimizationcited by 2
  31. 2025PointSplit: Towards On-device 3D Object Detection with Heterogeneous Low-power Acceleratorscited by 2
  32. 2025EdgeWisePersona: A Dataset for On-Device User Profiling from Natural Language Interactionscited by 2
  33. 2025Optimizing 3D Gaussian Splattering for Mobile GPUscited by 2
  34. 2026Parameter-Free Encoders Remain Viable for RDB Foundation Modelscited by 2
  35. 2021Morphable Detector for Object Detection on Demandcited by 1
  36. 2022AGO: Boosting Mobile AI Inference Performance by Removing Constraints on Graph Optimizationcited by 1
  37. 2023Enhancing Quantised End-to-End ASR Models via Personalisationcited by 1
  38. 2024Resource-Efficient Personal Large Language Models Fine-Tuning with Collaborative Edge Computingcited by 1

50 most cited of 65 All 65 (JSON)