# Intelligence at the Extreme Edge: A Survey on Reformable TinyML

arXiv: https://arxiv.org/abs/2204.00827 · HTML: https://mlsanity.com/paper/2204.00827/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 108 references that are in the map.
- First posted on arXiv 2 Apr 2022.

## Builds on (most cited first, 24 of 24)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [REMIND Your Neural Network to Prevent Catastrophic Forgetting](https://mlsanity.com/paper/1910.02509/) (2019)
- [TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems](https://mlsanity.com/paper/2010.08678/) (2020)
- [MLPerf Tiny Benchmark](https://mlsanity.com/paper/2106.07597/) (2021)
- [Hello Edge: Keyword Spotting on Microcontrollers](https://mlsanity.com/paper/1711.07128/) (2017)
- [Latent Replay for Real-Time Continual Learning](https://mlsanity.com/paper/1912.01100/) (2019)
- [Benchmarking TinyML Systems: Challenges and Direction](https://mlsanity.com/paper/2003.04821/) (2020)
- [TinyOL: TinyML with Online-Learning on Microcontrollers](https://mlsanity.com/paper/2103.08295/) (2021)
- [SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers](https://mlsanity.com/paper/1905.12107/) (2019)
- [Visual Wake Words Dataset](https://mlsanity.com/paper/1906.05721/) (2019)
- [A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays](https://mlsanity.com/paper/2110.10486/) (2021)
- [Widening Access to Applied Machine Learning with TinyML](https://mlsanity.com/paper/2106.04008/) (2021)
- [TinyML for Ubiquitous Edge AI](https://mlsanity.com/paper/2102.01255/) (2021)
- [AI for Next Generation Computing: Emerging Trends and Future Directions](https://mlsanity.com/paper/2203.04159/) (2022)
- [Robustifying the Deployment of tinyML Models for Autonomous mini-vehicles](https://mlsanity.com/paper/2007.00302/) (2020)
- [A Tiny CNN Architecture for Medical Face Mask Detection for Resource-Constrained Endpoints](https://mlsanity.com/paper/2011.14858/) (2020)
- [TinyFedTL: Federated Transfer Learning on Tiny Devices](https://mlsanity.com/paper/2110.01107/) (2021)
- [Firmware over-the-air programming techniques for IoT networks -- A survey](https://mlsanity.com/paper/2009.02260/) (2020)

## Built on by (most cited first, 13 of 13)

- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [TinyML: Tools, Applications, Challenges, and Future Research Directions](https://mlsanity.com/paper/2303.13569/) (2023)
- [On-device Online Learning and Semantic Management of TinyML Systems](https://mlsanity.com/paper/2405.07601/) (2024)
- [TinySV: Speaker Verification in TinyML with On-device Learning](https://mlsanity.com/paper/2406.01655/) (2024)
- [From Tiny Machine Learning to Tiny Deep Learning: A Survey](https://mlsanity.com/paper/2506.18927/) (2025)
- [Agentic TinyML for Intent-aware Handover in 6G Wireless Networks](https://mlsanity.com/paper/2508.09147/) (2025)
- [Efficient Adaptation of Deep Neural Networks for Semantic Segmentation in Space Applications](https://mlsanity.com/paper/2504.15991/) (2025)
- [A review of TinyML](https://mlsanity.com/paper/2211.04448/) (2022)
- [Accelerating TinyML Inference on Microcontrollers through Approximate Kernels](https://mlsanity.com/paper/2409.16815/) (2024)
- [Integration of TinyML and LargeML: A Survey of 6G and Beyond](https://mlsanity.com/paper/2505.15854/) (2025)
- [A Survey of TinyML Applications in Beekeeping for Hive Monitoring and Management](https://mlsanity.com/paper/2509.08822/) (2025)
- [What changes after deployment? A survey on On-device Learning in TinyML](https://mlsanity.com/paper/2605.31226/) (2026)
- [Unifying Data, Memory, and Compute Efficiency in LLM training: A Survey](https://mlsanity.com/paper/2606.10706/) (2026)

## Abstract (arXiv)

Tiny Machine Learning (TinyML) is an upsurging research field that proposes to democratize the use of Machine Learning and Deep Learning on highly energy-efficient frugal Microcontroller Units. Considering the general assumption that TinyML can only run inference, growing interest in the domain has led to work that makes them reformable, i.e., solutions that permit models to improve once deployed. This work presents a survey on reformable TinyML solutions with the proposal of a novel taxonomy. Here, the suitability of each hierarchical layer for reformability is discussed. Furthermore, we explore the workflow of TinyML and analyze the identified deployment schemes, available tools and the scarcely available benchmarking tools. Finally, we discuss how reformable TinyML can impact a few selected industrial areas and discuss the challenges and future directions.