# Bioformers: Embedding Transformers for Ultra-Low Power sEMG-based Gesture Recognition

arXiv: https://arxiv.org/abs/2203.12932 · HTML: https://mlsanity.com/paper/2203.12932/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [Stable Electromyographic Sequence Prediction During Movement Transitions using Temporal Convolutional Networks](https://mlsanity.com/paper/1901.02442/) (2019)

## Built on by (most cited first, 9 of 9)

- [PLiNIO: A User-Friendly Library of Gradient-based Methods for Complexity-aware DNN Optimization](https://mlsanity.com/paper/2307.09488/) (2023)
- [Marsellus: A Heterogeneous RISC-V AI-IoT End-Node SoC with 2-to-8b DNN Acceleration and 30%-Boost Adaptive Body Biasing](https://mlsanity.com/paper/2305.08415/) (2023)
- [A Tiny Transformer for Low-Power Arrhythmia Classification on Microcontrollers](https://mlsanity.com/paper/2402.10748/) (2024)
- [Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control](https://mlsanity.com/paper/2303.11860/) (2023)
- [Efficient Deep Learning Models for Privacy-preserving People Counting on Low-resolution Infrared Arrays](https://mlsanity.com/paper/2304.06059/) (2023)
- [Automating Versatile Time-Series Analysis with Tiny Transformers on Embedded FPGAs](https://mlsanity.com/paper/2505.17662/) (2025)
- [WaveFormer: A Lightweight Transformer Model for sEMG-based Gesture Recognition](https://mlsanity.com/paper/2506.11168/) (2025)
- [Optimization and Deployment of Deep Neural Networks for PPG-based Blood Pressure Estimation Targeting Low-power Wearables](https://mlsanity.com/paper/2409.07485/) (2024)
- [PGUDA: Pressure-Guided Unsupervised Domain Adaptation with Cross-Modal Knowledge Distillation for sEMG-Based Gesture Recognition](https://mlsanity.com/paper/2606.31349/) (2026)

## Abstract (arXiv)

Human-machine interaction is gaining traction in rehabilitation tasks, such as controlling prosthetic hands or robotic arms. Gesture recognition exploiting surface electromyographic (sEMG) signals is one of the most promising approaches, given that sEMG signal acquisition is non-invasive and is directly related to muscle contraction. However, the analysis of these signals still presents many challenges since similar gestures result in similar muscle contractions. Thus the resulting signal shapes are almost identical, leading to low classification accuracy. To tackle this challenge, complex neural networks are employed, which require large memory footprints, consume relatively high energy and limit the maximum battery life of devices used for classification. This work addresses this problem with the introduction of the Bioformers. This new family of ultra-small attention-based architectures approaches state-of-the-art performance while reducing the number of parameters and operations of 4.9X. Additionally, by introducing a new inter-subjects pre-training, we improve the accuracy of our best Bioformer by 3.39%, matching state-of-the-art accuracy without any additional inference cost. Deploying our best performing Bioformer on a Parallel, Ultra-Low Power (PULP) microcontroller unit (MCU), the GreenWaves GAP8, we achieve an inference latency and energy of 2.72 ms and 0.14 mJ, respectively, 8.0X lower than the previous state-of-the-art neural network, while occupying just 94.2 kB of memory.