# SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling

arXiv: https://arxiv.org/abs/2608.24334 · HTML: https://mlsanity.com/paper/2608.24334/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Builds on (most cited first, 26 of 26)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis](https://mlsanity.com/paper/2305.00976/) (2023)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [We are More than Our Joints: Predicting how 3D Bodies Move](https://mlsanity.com/paper/2012.00619/) (2020)
- [SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models](https://mlsanity.com/paper/2308.16692/) (2023)
- [Go to Zero: Towards Zero-shot Motion Generation with Million-scale Data](https://mlsanity.com/paper/2507.07095/) (2025)
- [Learning Complex 3D Human Self-Contact](https://mlsanity.com/paper/2012.10366/) (2020)
- [MotionGPT3: Human Motion as a Second Modality](https://mlsanity.com/paper/2506.24086/) (2025)
- [VersatileMotion: A Unified Framework for Motion Synthesis and Comprehension](https://mlsanity.com/paper/2411.17335/) (2024)
- [MotionBricks: Scalable Real-Time Motions with Modular Latent Generative Model and Smart Primitives](https://mlsanity.com/paper/2604.24833/) (2026)
- [UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation](https://mlsanity.com/paper/2603.22282/) (2026)
- [Next-Scale Autoregressive Models for Text-to-Motion Generation](https://mlsanity.com/paper/2604.03799/) (2026)
- [ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation](https://mlsanity.com/paper/2605.11704/) (2026)
- [Think Before You Move: Latent Motion Reasoning for Text-to-Motion Generation](https://mlsanity.com/paper/2512.24100/) (2025)
- [Pose-Guided Residual Refinement for Interpretable Text-to-Motion Generation and Editing](https://mlsanity.com/paper/2512.22464/) (2025)
- [Language-Guided Transformer Tokenizer for Human Motion Generation](https://mlsanity.com/paper/2602.08337/) (2026)
- [Beyond MoCap: Scaling Motion Tokenizers with Synthetic Human Motion for Generative Modeling](https://mlsanity.com/paper/2606.27547/) (2026)

## Built on by (most cited first, 1 of 1)

- [DualTrack: Synchronized speech-gesture generation via symmetric coupling of pretrained priors](https://mlsanity.com/paper/2609.36624/) (2026)

## Abstract (arXiv)

Discrete motion representations have substantially advanced autoregressive text-to-motion generation. However, most motion tokenizers are optimized for reconstruction and do not explicitly allocate capacity according to semantic role. Action-level meaning and fine-grained kinematic detail must therefore be encoded through the same reconstruction-driven hierarchy. We introduce SeMoCo, a semantic-first motion codec, together with a dual-axis motion generator for language-conditioned motion generation. Each motion token contains one semantic token and a residual sequence of kinematic tokens. The generator models semantic progression across time and autoregressively refines the residual entries. We also construct Omega-MotionVerse, a large-scale, multi-source human-motion dataset unified under the SOMA representation. Across the reported comparisons, SeMoCo achieves the best reconstruction accuracy among the compared codecs, while strong text-to-motion results demonstrate the effectiveness of its motion tokens for downstream generation.