# emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation

arXiv: https://arxiv.org/abs/2412.02725 · HTML: https://mlsanity.com/paper/2412.02725/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 91 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net](https://mlsanity.com/paper/1807.09441/) (2018)
- [HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction](https://mlsanity.com/paper/2203.01577/) (2022)
- [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [3D Hand Shape and Pose from Images in the Wild](https://mlsanity.com/paper/1902.03451/) (2019)
- [ContactPose: A Dataset of Grasps with Object Contact and Hand Pose](https://mlsanity.com/paper/2007.09545/) (2020)
- [GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB](https://mlsanity.com/paper/1712.01057/) (2017)
- [Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints](https://mlsanity.com/paper/2003.09282/) (2020)
- [Cross-modal Deep Variational Hand Pose Estimation](https://mlsanity.com/paper/1803.11404/) (2018)
- [Partially Observable Markov Decision Processes in Robotics: A Survey](https://mlsanity.com/paper/2209.10342/) (2022)
- [Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor](https://mlsanity.com/paper/1704.02201/) (2017)
- [MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control](https://mlsanity.com/paper/2205.13600/) (2022)
- [UmeTrack: Unified multi-view end-to-end hand tracking for VR](https://mlsanity.com/paper/2211.00099/) (2022)
- [BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis](https://mlsanity.com/paper/1704.02612/) (2017)
- [A Dataset of Relighted 3D Interacting Hands](https://mlsanity.com/paper/2310.17768/) (2023)
- [Teleoperation of Humanoid Robots: A Survey](https://mlsanity.com/paper/2301.04317/) (2023)
- [Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNs](https://mlsanity.com/paper/1606.07253/) (2016)
- [Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions](https://mlsanity.com/paper/1904.02619/) (2019)

## Built on by (most cited first, 22 of 22)

- [PhysioWave: A Multi-Scale Wavelet-Transformer for Physiological Signal Representation](https://mlsanity.com/paper/2506.10351/) (2025)
- [TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edge](https://mlsanity.com/paper/2512.15729/) (2025)
- [NeuroRVQ: Multi-Scale Biosignal Tokenization for Generative Foundation Models](https://mlsanity.com/paper/2510.13068/) (2025)
- [EgoEMG: A Multimodal Egocentric Dataset with Bilateral EMG and Vision for Hand Pose Estimation](https://mlsanity.com/paper/2605.05712/) (2026)
- [emg2tendon: From sEMG Signals to Tendon Control in Musculoskeletal Hands](https://mlsanity.com/paper/2508.08269/) (2025)
- [FLEX: A Largescale Multimodal, Multiview Dataset for Learning Structured Representations for Fitness Action Quality Assessment](https://mlsanity.com/paper/2506.03198/) (2025)
- [SplashNet: Split-and-Share Encoders for Accurate and Efficient Typing with Surface Electromyography](https://mlsanity.com/paper/2506.12356/) (2025)
- [Grounding Intelligence in Movement](https://mlsanity.com/paper/2507.02771/) (2025)
- [Visuo-Acoustic Hand Pose and Contact Estimation](https://mlsanity.com/paper/2508.00852/) (2025)
- [SPECTRE: Spectral Pre-training Embeddings with Cylindrical Temporal Rotary Position Encoding for Fine-Grained sEMG-Based Movement Decoding](https://mlsanity.com/paper/2512.22481/) (2025)
- [Typing Reinvented: Towards Hands-Free Input via sEMG](https://mlsanity.com/paper/2511.18213/) (2025)
- [AirGlove: Exploring Egocentric 3D Hand Tracking and Appearance Generalization for Sensing Gloves](https://mlsanity.com/paper/2602.05159/) (2026)
- [Bias in Surface Electromyography Features across a Demographically Diverse Cohort](https://mlsanity.com/paper/2604.14460/) (2026)
- [Decoding High-Dimensional Finger Motion from EMG Using Riemannian Features and RNNs](https://mlsanity.com/paper/2604.22499/) (2026)
- [Translating Signals to Languages for sEMG-Based Activity Recognition](https://mlsanity.com/paper/2605.22403/) (2026)
- [KinEMbed: Decoding Kinematics from Electromyography via Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2607.04820/) (2026)
- [MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching](https://mlsanity.com/paper/2608.26094/) (2026)
- [EMGBlend: Heterogeneity-Aware Self-Supervised Pretraining for Gesture and Force Decoding](https://mlsanity.com/paper/2609.25582/) (2026)
- [MyoCodec: A Streaming Neural Codec for Electromyography](https://mlsanity.com/paper/2609.36687/) (2026)
- [EFormer: Temporally Aligned Local Correction for Continuous sEMG-Based Hand Pose Tracking](https://mlsanity.com/paper/2609.38932/) (2026)
- [EMG-GPT: Predictive Pretraining on Residual-Quantized EMG Tokens for Hand Pose Estimation](https://mlsanity.com/paper/2610.05235/) (2026)
- [EVFormer: An Egocentric Vision-EMG Bidirectional Attention Model for Bimanual Hand Pose Estimation](https://mlsanity.com/paper/2610.06970/) (2026)

## Abstract (arXiv)

Hands are the primary means through which humans interact with the world. Reliable and always-available hand pose inference could yield new and intuitive control schemes for human-computer interactions, particularly in virtual and augmented reality. Computer vision is effective but requires one or multiple cameras and can struggle with occlusions, limited field of view, and poor lighting. Wearable wrist-based surface electromyography (sEMG) presents a promising alternative as an always-available modality sensing muscle activities that drive hand motion. However, sEMG signals are strongly dependent on user anatomy and sensor placement, and existing sEMG models have required hundreds of users and device placements to effectively generalize. To facilitate progress on sEMG pose inference, we introduce the emg2pose benchmark, the largest publicly available dataset of high-quality hand pose labels and wrist sEMG recordings. emg2pose contains 2kHz, 16 channel sEMG and pose labels from a 26-camera motion capture rig for 193 users, 370 hours, and 29 stages with diverse gestures - a scale comparable to vision-based hand pose datasets. We provide competitive baselines and challenging tasks evaluating real-world generalization scenarios: held-out users, sensor placements, and stages. emg2pose provides the machine learning community a platform for exploring complex generalization problems, holding potential to significantly enhance the development of sEMG-based human-computer interactions.