# Fast and Expressive Gesture Recognition using a Combination-Homomorphic Electromyogram Encoder

arXiv: https://arxiv.org/abs/2311.14675 · HTML: https://mlsanity.com/paper/2311.14675/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 53 references that are in the map.
- First posted on arXiv 30 Oct 2023.

## Builds on (most cited first, 17 of 17)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [An Overview of Deep Semi-Supervised Learning](https://mlsanity.com/paper/2006.05278/) (2020)
- [Compositional Generalization from First Principles](https://mlsanity.com/paper/2307.05596/) (2023)
- [Compositional Generalization in Image Captioning](https://mlsanity.com/paper/1909.04402/) (2019)
- [Enabling hand gesture customization on wrist-worn devices](https://mlsanity.com/paper/2203.15239/) (2022)
- [Transfer Learning of an Ensemble of DNNs for SSVEP BCI Spellers without User-Specific Training](https://mlsanity.com/paper/2209.01511/) (2022)
- [Transfer Learning on Electromyography (EMG) Tasks: Approaches and Beyond](https://mlsanity.com/paper/2210.06295/) (2022)
- [AutoTransfer: Subject Transfer Learning with Censored Representations on Biosignals Data](https://mlsanity.com/paper/2112.09796/) (2021)
- [User Training with Error Augmentation for Electromyogram-based Gesture Classification](https://mlsanity.com/paper/2309.07289/) (2023)
- [A Multi-label Classification Approach to Increase Expressivity of EMG-based Gesture Recognition](https://mlsanity.com/paper/2309.12217/) (2023)

## Abstract (arXiv)

We study the task of gesture recognition from electromyography (EMG), with the goal of enabling expressive human-computer interaction at high accuracy, while minimizing the time required for new subjects to provide calibration data. To fulfill these goals, we define combination gestures consisting of a direction component and a modifier component. New subjects only demonstrate the single component gestures and we seek to extrapolate from these to all possible single or combination gestures. We extrapolate to unseen combination gestures by combining the feature vectors of real single gestures to produce synthetic training data. This strategy allows us to provide a large and flexible gesture vocabulary, while not requiring new subjects to demonstrate combinatorially many example gestures. We pre-train an encoder and a combination operator using self-supervision, so that we can produce useful synthetic training data for unseen test subjects. To evaluate the proposed method, we collect a real-world EMG dataset, and measure the effect of augmented supervision against two baselines: a partially-supervised model trained with only single gesture data from the unseen subject, and a fully-supervised model trained with real single and real combination gesture data from the unseen subject. We find that the proposed method provides a dramatic improvement over the partially-supervised model, and achieves a useful classification accuracy that in some cases approaches the performance of the fully-supervised model.