# emg2qwerty: A Large Dataset with Baselines for Touch Typing using Surface Electromyography

arXiv: https://arxiv.org/abs/2410.20081 · HTML: https://mlsanity.com/paper/2410.20081/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 68 references that are in the map.
- First posted on arXiv 26 Oct 2024.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures](https://mlsanity.com/paper/1911.08460/) (2019)
- [SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG](https://mlsanity.com/paper/2206.01323/) (2022)
- [wav2letter++: The Fastest Open-source Speech Recognition System](https://mlsanity.com/paper/1812.07625/) (2018)
- [Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions](https://mlsanity.com/paper/1904.02619/) (2019)
- [Scaling Up Online Speech Recognition Using ConvNets](https://mlsanity.com/paper/2001.09727/) (2020)
- [Estimating Phoneme Class Conditional Probabilities from Raw Speech Signal using Convolutional Neural Networks](https://mlsanity.com/paper/1304.1018/) (2013)
- [Plug-and-Play Stability for Intracortical Brain-Computer Interfaces: A One-Year Demonstration of Seamless Brain-to-Text Communication](https://mlsanity.com/paper/2311.03611/) (2023)
- [Latent Alignment with Deep Set EEG Decoders](https://mlsanity.com/paper/2311.17968/) (2023)
- [Common Synaptic Input, Synergies, and Size Principle: Control of Spinal Motor Neurons for Movement Generation](https://mlsanity.com/paper/2208.02818/) (2022)
- [EEG and EMG dataset for the detection of errors introduced by an active orthosis device](https://mlsanity.com/paper/2305.11996/) (2023)

## Built on by (most cited first, 18 of 18)

- [Unlocking Non-Invasive Brain-to-Text](https://mlsanity.com/paper/2505.13446/) (2025)
- [Neural Codecs as Biosignal Tokenizers](https://mlsanity.com/paper/2510.09095/) (2025)
- [PhysioWave: A Multi-Scale Wavelet-Transformer for Physiological Signal Representation](https://mlsanity.com/paper/2506.10351/) (2025)
- [NeuroRVQ: Multi-Scale Biosignal Tokenization for Generative Foundation Models](https://mlsanity.com/paper/2510.13068/) (2025)
- [Accurate Decoding of Natural Sentences from Non-Invasive Brain Recordings](https://mlsanity.com/paper/2608.18114/) (2026)
- [Scaling and Distilling Transformer Models for sEMG](https://mlsanity.com/paper/2507.22094/) (2025)
- [emg2speech: Synthesizing speech from electromyography using self-supervised speech models](https://mlsanity.com/paper/2510.23969/) (2025)
- [SplashNet: Split-and-Share Encoders for Accurate and Efficient Typing with Surface Electromyography](https://mlsanity.com/paper/2506.12356/) (2025)
- [SPECTRE: Spectral Pre-training Embeddings with Cylindrical Temporal Rotary Position Encoding for Fine-Grained sEMG-Based Movement Decoding](https://mlsanity.com/paper/2512.22481/) (2025)
- [EMGFlow: Robust and Efficient Surface Electromyography Synthesis via Flow Matching](https://mlsanity.com/paper/2604.13685/) (2026)
- [Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections](https://mlsanity.com/paper/2510.22655/) (2025)
- [SASG-DA: Sparse-Aware Semantic-Guided Diffusion Augmentation For Myoelectric Gesture Recognition](https://mlsanity.com/paper/2511.08344/) (2025)
- [Typing Reinvented: Towards Hands-Free Input via sEMG](https://mlsanity.com/paper/2511.18213/) (2025)
- [DANCE: Detect and Classify Events in EEG](https://mlsanity.com/paper/2605.10688/) (2026)
- [Integrated Sensing and Communications for Real-time Avatar Control in XR over 5G](https://mlsanity.com/paper/2606.23771/) (2026)
- [EMGBlend: Heterogeneity-Aware Self-Supervised Pretraining for Gesture and Force Decoding](https://mlsanity.com/paper/2609.25582/) (2026)
- [MyoCodec: A Streaming Neural Codec for Electromyography](https://mlsanity.com/paper/2609.36687/) (2026)
- [LiteEMG-FM: An Efficient and Deployable Foundation Model for Robust EMG Sensing](https://mlsanity.com/paper/2610.02497/) (2026)

## Abstract (arXiv)

Surface electromyography (sEMG) non-invasively measures signals generated by muscle activity with sufficient sensitivity to detect individual spinal neurons and richness to identify dozens of gestures and their nuances. Wearable wrist-based sEMG sensors have the potential to offer low friction, subtle, information rich, always available human-computer inputs. To this end, we introduce emg2qwerty, a large-scale dataset of non-invasive electromyographic signals recorded at the wrists while touch typing on a QWERTY keyboard, together with ground-truth annotations and reproducible baselines. With 1,135 sessions spanning 108 users and 346 hours of recording, this is the largest such public dataset to date. These data demonstrate non-trivial, but well defined hierarchical relationships both in terms of the generative process, from neurons to muscles and muscle combinations, as well as in terms of domain shift across users and user sessions. Applying standard modeling techniques from the closely related field of Automatic Speech Recognition (ASR), we show strong baseline performance on predicting key-presses using sEMG signals alone. We believe the richness of this task and dataset will facilitate progress in several problems of interest to both the machine learning and neuroscientific communities. Dataset and code can be accessed at https://github.com/facebookresearch/emg2qwerty.