mlsanityData to 14 Oct 2026

Tiny Machine Learning: Progress and Futures

arXiv 2403.19076 ↗First posted 28 Mar 2024cs.LG cs.AI cs.CVcited by 40

Not on the map. Why?

Abstract (arXiv)

Tiny Machine Learning (TinyML) is a new frontier of machine learning. By squeezing deep learning models into billions of IoT devices and microcontrollers (MCUs), we expand the scope of AI applications and enable ubiquitous intelligence. However, TinyML is challenging due to hardware constraints: the tiny memory resource makes it difficult to hold deep learning models designed for cloud and mobile platforms. There is also limited compiler and inference engine support for bare-metal devices. Therefore, we need to co-design the algorithm and system stack to enable TinyML. In this review, we will first discuss the definition, challenges, and applications of TinyML. We then survey the recent progress in TinyML and deep learning on MCUs. Next, we will introduce MCUNet, showing how we can achieve ImageNet-scale AI applications on IoT devices with system-algorithm co-design. We will further extend the solution from inference to training and introduce tiny on-device training techniques. Finally, we present future directions in this area. Today's large model might be tomorrow's tiny model. The scope of TinyML should evolve and adapt over time.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 103 of its 154 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

103 of 154 references.

References · 103

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  10. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  11. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  12. 2018YOLOv3: An Incremental Improvementcited by 2,659
38 more
  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  2. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  3. 2019Searching for MobileNetV3cited by 1,994
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  5. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  9. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  10. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  11. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  12. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  13. 2020Designing Network Design Spacescited by 960
  14. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  15. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  16. 2019Relational Knowledge Distillationcited by 889
  17. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  18. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  19. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  20. 2021Towards Total Recall in Industrial Anomaly Detectioncited by 776
  21. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  22. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  23. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  24. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  25. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  26. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  27. 2018Do Better ImageNet Models Transfer Better?cited by 565
  28. 2017Large Batch Training of Convolutional Networkscited by 535
  29. 2019Similarity-Preserving Knowledge Distillationcited by 516
  30. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  31. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  32. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  33. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 424
  34. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  35. 2022Decoupled Knowledge Distillationcited by 388
  36. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  37. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  38. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 316

50 most cited of 103 All 103 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 20; 2026: 12.

40 papers.

Sort:

By area:

  1. 2024On-Device Language Models: A Comprehensive Reviewcited by 104
  2. 2024Streaming Deep Reinforcement Learning Finally Workscited by 25
  3. 2025A Survey on Collaborative Mechanisms Between Large and Small Language Modelscited by 14
  4. 2025TeleSparse: Practical Privacy-Preserving Verification of Deep Neural Networkscited by 9
  5. 2024Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensorscited by 7
  6. 2024On-Device Training of Fully Quantized Deep Neural Networks on Cortex-M Microcontrollerscited by 5
  7. 2025Can LLMs Revolutionize the Design of Explainable and Efficient TinyML Models?cited by 5
  8. 2025From Tiny Machine Learning to Tiny Deep Learning: A Surveycited by 5
  9. 2024Revisiting Matrix Sketching in Linear Bandits: Achieving Sublinear Regret via Dyadic Block Sketchingcited by 4
  10. 2025From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devicescited by 4
  11. 2025Cycle Training with Semi-Supervised Domain Adaptation: Bridging Accuracy and Efficiency for Real-Time Mobile Scene Detectioncited by 3
  12. 2025A Multicore and Edge TPU-Accelerated Multimodal TinyML System for Livestock Behavior Recognitioncited by 3
28 more
  1. 2025Rounding-Guided Backdoor Injection in Deep Learning Model Quantizationcited by 3
  2. 2024A Progressive Framework of Vision-language Knowledge Distillation and Alignment for Multilingual Scenecited by 2
  3. 2024TinyML NLP Scheme for Semantic Wireless Sentiment Classification with Privacy Preservationcited by 2
  4. 2025Edge Intelligence for Wildlife Conservation: Real-Time Hornbill Call Classification Using TinyMLcited by 2
  5. 2025Hybrid Knowledge Transfer through Attention and Logit Distillation for On-Device Vision Systems in Agricultural IoTcited by 2
  6. 2025Edge Artificial Intelligence: A Systematic Review of Evolution, Taxonomic Frameworks, and Future Horizonscited by 2
  7. 2024LightCL: Compact Continual Learning with Low Memory Footprint For Edge Devicecited by 1
  8. 2025Onboard Optimization and Learning: A Surveycited by 1
  9. 2025Designing Object Detection Models for TinyML: Foundations, Comparative Analysis, Challenges, and Emerging Solutionscited by 1
  10. 2025AI Compute Architecture and Evolution Trendscited by 1
  11. 2025AdaBet: Gradient-free Layer Selection for Efficient Training of Deep Neural Networkscited by 1
  12. 2025RISC-V Based TinyML Accelerator for Depthwise Separable Convolutions in Edge AIcited by 1
  13. 2026The Weight of a Bit: EMFI Sensitivity Analysis of Embedded Deep Learning Modelscited by 1
  14. 2026TrainDeeploy: Hardware-Accelerated Parameter-Efficient Fine-Tuning of Small Transformer Models at the Extreme Edgecited by 1
  15. 2025EmbBERT: Attention Under 2 MB Memorycited by 0
  16. 2025Compound and Parallel Modes of Tropical Convolutional Neural Networkscited by 0
  17. 2025Integration of TinyML and LargeML: A Survey of 6G and Beyondcited by 0
  18. 2025Beyond ImageNet: Understanding Cross-Dataset Robustness of Lightweight Vision Modelscited by 0
  19. 2026Towards Resilient Intrusion Detection in CubeSats: Challenges, TinyML Solutions, and Future Directionscited by 0
  20. 2026BioTrain: Sub-MB, Sub-50mW On-Device Fine-Tuning for Edge-AI on Biosignalscited by 0
  21. 2026On-Device Vision Training, Deployment, and Inference on a Thumb-Sized Microcontrollercited by 0
  22. 2026BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deploymentcited by 0
  23. 2026One Channel to Rule Them All: Rethinking Input Representation for Visual Place Recognitioncited by 0
  24. 2026BayaHAR: Lightweight Bayesian Few-Shot User Adaptation for On-Device Personalized Human Activity Recognitioncited by 0
  25. 2026tinyDSM: A Framework for Skill Modeling and Development for Resource-Constrained Millirobotscited by 0
  26. 2026Adaptive AI: Energy Efficient Multi-exit TinyML on Intelligent Vision Systems at the Edgecited by 0
  27. 2026Rank-Reliable Teacher-Guided Fitness Approximation for Expensive Evolutionary Optimization: A TinyML Architecture Search Studycited by 0
  28. 2026Right In-Place (RiP) Convolution: A Simple, General, and Near-Optimal Strategy for Memory-Efficient CNN Inferencecited by 0