mlsanityData to 14 Oct 2026

A Machine Learning-oriented Survey on Tiny Machine Learning

arXiv 2309.11932 ↗First posted 21 Sep 2023cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

The emergence of Tiny Machine Learning (TinyML) has positively revolutionized the field of Artificial Intelligence by promoting the joint design of resource-constrained IoT hardware devices and their learning-based software architectures. TinyML carries an essential role within the fourth and fifth industrial revolutions in helping societies, economies, and individuals employ effective AI-infused computing technologies (e.g., smart cities, automotive, and medical robotics). Given its multidisciplinary nature, the field of TinyML has been approached from many different angles: this comprehensive survey wishes to provide an up-to-date overview focused on all the learning algorithms within TinyML-based solutions. The survey is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodological flow, allowing for a systematic and complete literature survey. In particular, firstly we will examine the three different workflows for implementing a TinyML-based system, i.e., ML-oriented, HW-oriented, and co-design. Secondly, we propose a taxonomy that covers the learning panorama under the TinyML lens, examining in detail the different families of model optimization and design, as well as the state-of-the-art learning techniques. Thirdly, this survey will present the distinct features of hardware devices and software tools that represent the current state-of-the-art for TinyML intelligent edge applications. Finally, we discuss the challenges and future directions.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 79 of its 176 references that are in the map. First posted on arXiv 21 Sep 2023.

Builds on

79 of 176 references.

References · 79

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  5. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  6. 2020Knowledge Distillation: A Surveycited by 1,603
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  8. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  9. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  10. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  11. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
38 more
  1. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  2. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  3. 2018Rethinking the Value of Network Pruningcited by 599
  4. 2020What is the State of Neural Network Pruning?cited by 583
  5. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  6. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  7. 2015Attention-Based Models for Speech Recognitioncited by 401
  8. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 379
  9. 2019Attention on Attention for Image Captioningcited by 289
  10. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
  11. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 254
  12. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 221
  13. 2019MLPerf Inference Benchmarkcited by 199
  14. 2020The Computational Limits of Deep Learningcited by 193
  15. 2020TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systemscited by 165
  16. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 149
  17. 2020Regularizing Class-wise Predictions via Self-knowledge Distillationcited by 144
  18. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
  19. 2021MLPerf Tiny Benchmarkcited by 124
  20. 2018CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUscited by 121
  21. 2020DROCC: Deep Robust One-Class Classificationcited by 121
  22. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 119
  23. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 117
  24. 2019MLPerf Training Benchmarkcited by 115
  25. 2021General Instance Distillation for Object Detectioncited by 114
  26. 2019Attention in Natural Language Processingcited by 101
  27. 2020Benchmarking TinyML Systems: Challenges and Directioncited by 91
  28. 2020A survey on modern trainable activation functionscited by 89
  29. 2018NVIDIA Tensor Core Programmability, Performance & Precisioncited by 79
  30. 2019Pad'e Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networkscited by 67
  31. 2021TinyOL: TinyML with Online-Learning on Microcontrollerscited by 65
  32. 2022A General Survey on Attention Mechanisms in Deep Learningcited by 64
  33. 2020CompRess: Self-Supervised Learning by Compressing Representationscited by 62
  34. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  35. 2020μNAS: Constrained Neural Architecture Search for Microcontrollerscited by 55
  36. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 54
  37. 2019Benchmarking TPU, GPU, and CPU Platforms for Deep Learningcited by 52
  38. 2020Methods for Pruning Deep Neural Networkscited by 49

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 9; 2026: 9.

27 papers.

Sort:

By area:

  1. 2024Review of Generative AI Methods in Cybersecuritycited by 9
  2. 2024Leveraging Latent Diffusion Models for Training-Free In-Distribution Data Augmentation for Surface Defect Detectioncited by 8
  3. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 7
  4. 2025From Tiny Machine Learning to Tiny Deep Learning: A Surveycited by 5
  5. 2026Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Reviewcited by 4
  6. 2024TinyML Security: Exploring Vulnerabilities in Resource-Constrained Machine Learning Systemscited by 3
  7. 2025Consolidating TinyML Lifecycle with Large Language Models: Reality, Illusion, or Opportunity?cited by 3
  8. 2026TinyML Enhances CubeSat Mission Capabilitiescited by 3
  9. 2025EdgeMark: An Automation and Benchmarking System for Embedded Artificial Intelligence Toolscited by 2
  10. 2025KairosAD: A SAM-Based Model for Industrial Anomaly Detection on Embedded Devicescited by 2
  11. 2026TinyGLASS: Real-Time Self-Supervised In-Sensor Anomaly Detectioncited by 2
  12. 2024Enhancing Split Computing and Early Exit Applications through Predefined Sparsitycited by 1
15 more
  1. 2024Multi-Camera Industrial Open-Set Person Re-Identification and Trackingcited by 1
  2. 2024SpokeN-100: A Cross-Lingual Benchmarking Dataset for The Classification of Spoken Numbers in Different Languagescited by 0
  3. 2024Towards Transparent and Efficient Anomaly Detection in Industrial Processes through ExIFFIcited by 0
  4. 2024On-device Anomaly Detection in Conveyor Belt Operationscited by 0
  5. 2024Learning based Ge'ez character handwritten recognitioncited by 0
  6. 2025How to Train Your Metamorphic Deep Neural Networkcited by 0
  7. 2025Integration of TinyML and LargeML: A Survey of 6G and Beyondcited by 0
  8. 2025CURA: Size Isnt All You Need -- A Compact Universal Architecture for On-Device Intelligencecited by 0
  9. 2025BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Trainingcited by 0
  10. 2026Efficient Sensor Fusion for Gesture Recognition on Resource-Constrained Devicescited by 0
  11. 2026Exploiting In-Sensor Computing for Energy-Efficient Earth Observationcited by 0
  12. 2026Event-Based Vision in Space: Applications, Trends, and Future Directionscited by 0
  13. 2026LiZAD: A Lightweight Zero-Shot Anomaly Detection Framework for Industrial Manufacturingcited by 0
  14. 2026Filling the Pareto-Optimal Front for Affordance Segmentation on Embedded Devices Using RGB-D Camerascited by 0
  15. 2026On the Instance Hardness as a Decision Criterion in TinyML Systemscited by 0