# A Machine Learning-oriented Survey on Tiny Machine Learning

arXiv: https://arxiv.org/abs/2309.11932 · HTML: https://mlsanity.com/paper/2309.11932/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 79 of its 176 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 30 of 79)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [Attention on Attention for Image Captioning](https://mlsanity.com/paper/1908.06954/) (2019)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions](https://mlsanity.com/paper/2006.02903/) (2020)
- [Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures](https://mlsanity.com/paper/1607.03250/) (2016)
- [MLPerf Inference Benchmark](https://mlsanity.com/paper/1911.02549/) (2019)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems](https://mlsanity.com/paper/2010.08678/) (2020)
- [Structured Pruning of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1512.08571/) (2015)
- [Regularizing Class-wise Predictions via Self-knowledge Distillation](https://mlsanity.com/paper/2003.13964/) (2020)
- [A Survey on Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2008.10937/) (2020)

## Built on by (most cited first, 27 of 27)

- [Review of Generative AI Methods in Cybersecurity](https://mlsanity.com/paper/2403.08701/) (2024)
- [Leveraging Latent Diffusion Models for Training-Free In-Distribution Data Augmentation for Surface Defect Detection](https://mlsanity.com/paper/2407.03961/) (2024)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [From Tiny Machine Learning to Tiny Deep Learning: A Survey](https://mlsanity.com/paper/2506.18927/) (2025)
- [Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Review](https://mlsanity.com/paper/2603.08725/) (2026)
- [TinyML Security: Exploring Vulnerabilities in Resource-Constrained Machine Learning Systems](https://mlsanity.com/paper/2411.07114/) (2024)
- [Consolidating TinyML Lifecycle with Large Language Models: Reality, Illusion, or Opportunity?](https://mlsanity.com/paper/2501.12420/) (2025)
- [TinyML Enhances CubeSat Mission Capabilities](https://mlsanity.com/paper/2603.20174/) (2026)
- [EdgeMark: An Automation and Benchmarking System for Embedded Artificial Intelligence Tools](https://mlsanity.com/paper/2502.01700/) (2025)
- [KairosAD: A SAM-Based Model for Industrial Anomaly Detection on Embedded Devices](https://mlsanity.com/paper/2505.24334/) (2025)
- [TinyGLASS: Real-Time Self-Supervised In-Sensor Anomaly Detection](https://mlsanity.com/paper/2603.16451/) (2026)
- [Enhancing Split Computing and Early Exit Applications through Predefined Sparsity](https://mlsanity.com/paper/2407.11763/) (2024)
- [Multi-Camera Industrial Open-Set Person Re-Identification and Tracking](https://mlsanity.com/paper/2409.03879/) (2024)
- [SpokeN-100: A Cross-Lingual Benchmarking Dataset for The Classification of Spoken Numbers in Different Languages](https://mlsanity.com/paper/2403.09753/) (2024)
- [Towards Transparent and Efficient Anomaly Detection in Industrial Processes through ExIFFI](https://mlsanity.com/paper/2405.01158/) (2024)
- [On-device Anomaly Detection in Conveyor Belt Operations](https://mlsanity.com/paper/2411.10729/) (2024)
- [Learning based Ge'ez character handwritten recognition](https://mlsanity.com/paper/2411.13350/) (2024)
- [How to Train Your Metamorphic Deep Neural Network](https://mlsanity.com/paper/2505.05510/) (2025)
- [Integration of TinyML and LargeML: A Survey of 6G and Beyond](https://mlsanity.com/paper/2505.15854/) (2025)
- [CURA: Size Isnt All You Need -- A Compact Universal Architecture for On-Device Intelligence](https://mlsanity.com/paper/2509.24601/) (2025)
- [BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training](https://mlsanity.com/paper/2512.04189/) (2025)
- [Efficient Sensor Fusion for Gesture Recognition on Resource-Constrained Devices](https://mlsanity.com/paper/2605.13462/) (2026)
- [Exploiting In-Sensor Computing for Energy-Efficient Earth Observation](https://mlsanity.com/paper/2606.01271/) (2026)
- [Event-Based Vision in Space: Applications, Trends, and Future Directions](https://mlsanity.com/paper/2606.01280/) (2026)
- [LiZAD: A Lightweight Zero-Shot Anomaly Detection Framework for Industrial Manufacturing](https://mlsanity.com/paper/2607.01949/) (2026)
- [Filling the Pareto-Optimal Front for Affordance Segmentation on Embedded Devices Using RGB-D Cameras](https://mlsanity.com/paper/2607.28293/) (2026)
- [On the Instance Hardness as a Decision Criterion in TinyML Systems](https://mlsanity.com/paper/2608.29913/) (2026)

## Abstract (arXiv)

The emergence of Tiny Machine Learning (TinyML) has positively revolutionized the field of Artificial Intelligence by promoting the joint design of resource-constrained IoT hardware devices and their learning-based software architectures. TinyML carries an essential role within the fourth and fifth industrial revolutions in helping societies, economies, and individuals employ effective AI-infused computing technologies (e.g., smart cities, automotive, and medical robotics). Given its multidisciplinary nature, the field of TinyML has been approached from many different angles: this comprehensive survey wishes to provide an up-to-date overview focused on all the learning algorithms within TinyML-based solutions. The survey is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodological flow, allowing for a systematic and complete literature survey. In particular, firstly we will examine the three different workflows for implementing a TinyML-based system, i.e., ML-oriented, HW-oriented, and co-design. Secondly, we propose a taxonomy that covers the learning panorama under the TinyML lens, examining in detail the different families of model optimization and design, as well as the state-of-the-art learning techniques. Thirdly, this survey will present the distinct features of hardware devices and software tools that represent the current state-of-the-art for TinyML intelligent edge applications. Finally, we discuss the challenges and future directions.