mlsanityData to 14 Oct 2026

Scaling and Distilling Transformer Models for sEMG

arXiv 2507.22094 ↗First posted 29 Jul 2025eess.AS cs.AI cs.HCcited by 3

Not on the map. Why?

Abstract (arXiv)

Surface electromyography (sEMG) signals offer a promising avenue for developing innovative human-computer interfaces by providing insights into muscular activity. However, the limited volume of training data and computational constraints during deployment have restricted the investigation of scaling up the model size for solving sEMG tasks. In this paper, we demonstrate that vanilla transformer models can be effectively scaled up on sEMG data and yield improved cross-user performance up to 110M parameters, surpassing the model size regime investigated in other sEMG research (usually <10M parameters). We show that >100M-parameter models can be effectively distilled into models 50x smaller with minimal loss of performance (<1.5% absolute). This results in efficient and expressive models suitable for complex real-time sEMG tasks in real-world environments.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 65 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

22 of 65 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  6. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  7. 2019Relational Knowledge Distillationcited by 889
  8. 2022Efficiently Scaling Transformer Inferencecited by 539
  9. 2019Similarity-Preserving Knowledge Distillationcited by 516
  10. 2022Decoupled Knowledge Distillationcited by 388
  11. 2020Cross-Layer Distillation with Semantic Calibrationcited by 143
  12. 2020Understanding and Improving Knowledge Distillationcited by 83
10 more
  1. 2023DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Modelscited by 36
  2. 2019Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutionscited by 27
  3. 2022FitHuBERT: Going Thinner and Deeper for Knowledge Distillation of Speech Self-Supervised Learningcited by 25
  4. 2024emg2qwerty: A Large Dataset with Baselines for Touch Typing using Surface Electromyographycited by 18
  5. 2022TraHGR: Transformer for Hand Gesture Recognition via ElectroMyographycited by 10
  6. 2021TEMGNet: Deep Transformer-based Decoding of Upperlimb sEMG for Hand Gestures Recognitioncited by 7
  7. 2024EMGBench: Benchmarking Out-of-Distribution Generalization and Adaptation for Electromyographycited by 7
  8. 2022Exploring Effective Distillation of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 3
  9. 2023Electromyography Signal Classification Using Deep Learningcited by 1
  10. 2023Knowledge Distilled Ensemble Model for sEMG-based Silent Speech Interfacecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026EgoEMG: A Multimodal Egocentric Dataset with Bilateral EMG and Vision for Hand Pose Estimationcited by 3
  2. 2026FatigueFormer: Static-Temporal Feature Fusion for Robust sEMG-Based Muscle Fatigue Recognitioncited by 0
  3. 2026EVFormer: An Egocentric Vision-EMG Bidirectional Attention Model for Bimanual Hand Pose Estimationcited by 0