mlsanityData to 14 Oct 2026

MyoSem: Aligning Electromyography with Natural-Language Action Semantics

arXiv 2606.00174 ↗First posted 29 May 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Electromyography (EMG) records muscle activation for hand gesture recognition, prosthetic control, and wearable interaction. Existing EMG methods usually formulate hand action understanding as prediction over fixed gesture labels, leaving EMG signals and action-semantic text unaligned in a common feature space and limiting bidirectional querying between signals and hand-action descriptions. We present MyoSem, an EMG--action semantic alignment framework that maps hand EMG and multi-view natural-language action descriptions into a shared feature space for retrieval. MyoSem builds semantic prototypes from multi-view descriptions, encodes EMG with local activation-aware representations, and uses semantic queries to aggregate local EMG evidence for bidirectional EMG--text retrieval. We evaluate MyoSem under seven protocols on NinaPro DB2, EMG2Pose, and NinaPro DB2rightarrowDB3, covering random splits, user shifts, held-out action classes, and able-bodied-to-amputee transfer. Across this evaluation, MyoSem achieves the strongest overall bidirectional retrieval performance among the evaluated methods. Ablations and representation analyses show how semantic prototypes, activation-aware encoding, and semantic queries shape the learned space. Overall, MyoSem moves EMG hand action understanding from fixed-label recognition toward language-queryable semantic retrieval.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 51 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

24 of 51 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  8. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  9. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  10. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  11. 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
  12. 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 266
12 more
  1. 2022Chinese CLIP: Contrastive Vision-Language Pretraining in Chinesecited by 109
  2. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  3. 2025SensorLM: Learning the Language of Wearable Sensorscited by 49
  4. 2025Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Modelcited by 36
  5. 2025EEG-CLIP : Learning EEG representations from natural language descriptionscited by 9
  6. 2024EMGBench: Benchmarking Out-of-Distribution Generalization and Adaptation for Electromyographycited by 7
  7. 2025Teaching Time Series to See and Speak: Forecasting with Aligned Visual and Textual Perspectivescited by 7
  8. 2025Can LLMs Understand Unvoiced Speech? Exploring EMG-to-Text Conversion with LLMscited by 6
  9. 2024BrainGPT: Unleashing the Potential of EEG Generalist Foundation Model by Autoregressive Pre-trainingcited by 4
  10. 2025Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokenscited by 4
  11. 2025CPEP: Contrastive Pose-EMG Pre-training Enhances Gesture Generalization on EMG Signalscited by 2
  12. 2025ReactEMG: Stable, Low-Latency Intent Detection from sEMG via Masked Modelingcited by 1

Built on by

Not cited yet.