# Optimising TinyML with Quantization and Distillation of Transformer and Mamba Models for Indoor Localisation on Edge Devices arXiv: https://arxiv.org/abs/2412.09289 ยท HTML: https://mlsanity.com/paper/2412.09289/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). - Builds on 20 of its 43 references that are in the map. - First posted on arXiv 12 Dec 2024. ## Placed in - State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/) ## Builds on (most cited first, 20 of 20) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023) - [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014) - [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017) - [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021) - [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020) - [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017) - [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021) - [Less is More: Task-aware Layer-wise Distillation for Language Model Compression](https://mlsanity.com/paper/2210.01351/) (2022) - [TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting](https://mlsanity.com/paper/2403.09898/) (2024) - [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024) - [DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer](https://mlsanity.com/paper/1707.01220/) (2017) - [A Survey of Techniques for Optimizing Transformer Inference](https://mlsanity.com/paper/2307.07982/) (2023) - [Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation](https://mlsanity.com/paper/2306.10687/) (2023) - [Saturn: Sample-efficient Generative Molecular Design using Memory Manipulation](https://mlsanity.com/paper/2405.17066/) (2024) - [Greedy-layer Pruning: Speeding up Transformer Models for Natural Language Processing](https://mlsanity.com/paper/2105.14839/) (2021) - [EPSD: Early Pruning with Self-Distillation for Efficient Model Compression](https://mlsanity.com/paper/2402.00084/) (2024) - [Multimodal Indoor Localisation in Parkinson's Disease for Detecting Medication Use: Observational Pilot Study in a Free-Living Setting](https://mlsanity.com/paper/2308.02419/) (2023) - [Detecting Signatures of Early-stage Dementia with Behavioural Models Derived from Sensor Data](https://mlsanity.com/paper/2007.03615/) (2020) ## Built on by (most cited first, 5 of 5) - [Quantizing Small-Scale State-Space Models for Edge AI](https://mlsanity.com/paper/2506.12480/) (2025) - [MambaLite-Micro: Memory-Optimized Mamba Inference on MCUs](https://mlsanity.com/paper/2509.05488/) (2025) - [A Survey of TinyML Applications in Beekeeping for Hive Monitoring and Management](https://mlsanity.com/paper/2509.08822/) (2025) - [Lightweight Transformer Architectures for Edge Devices in Real-Time Applications](https://mlsanity.com/paper/2601.03290/) (2026) - [TT-SEAL: TTD-Aware Selective Encryption for Adversarially-Robust and Low-Latency Edge AI](https://mlsanity.com/paper/2602.22238/) (2026) ## Abstract (arXiv) This paper proposes small and efficient machine learning models (TinyML) for resource-constrained edge devices, specifically for on-device indoor localisation. Typical approaches for indoor localisation rely on centralised remote processing of data transmitted from lower powered devices such as wearables. However, there are several benefits for moving this to the edge device itself, including increased battery life, enhanced privacy, reduced latency and lowered operational costs, all of which are key for common applications such as health monitoring. The work focuses on model compression techniques, including quantization and knowledge distillation, to significantly reduce the model size while maintaining high predictive performance. We base our work on a large state-of-the-art transformer-based model and seek to deploy it within low-power MCUs. We also propose a state-space-based architecture using Mamba as a more compact alternative to the transformer. Our results show that the quantized transformer model performs well within a 64 KB RAM constraint, achieving an effective balance between model size and localisation precision. Additionally, the compact Mamba model has strong performance under even tighter constraints, such as a 32 KB of RAM, without the need for model compression, making it a viable option for more resource-limited environments. We demonstrate that, through our framework, it is feasible to deploy advanced indoor localisation models onto low-power MCUs with restricted memory limitations. The application of these TinyML models in healthcare has the potential to revolutionize patient monitoring by providing accurate, real-time location data while minimizing power consumption, increasing data privacy, improving latency and reducing infrastructure costs.