DigitCode: Symbolic Tokenization of Hand Motion by Anatomical Units
Not on the map. Why?
Abstract (arXiv)
Hand motion carries the finest-grained information in human activity, yet the representations behind hand generation, understanding, and robot learning are overwhelmingly continuous--joint angles or MANO parameters. These are accurate but unstructured: a finger cannot be indexed or edited as a symbol, and nothing marks a pose as anatomically valid. Discrete symbolic representations supply exactly this structure, and Hand Labanotation (HL) has shown they are feasible for the hand, writing motion as a T x 40 grid of one fixed direction symbol per bone. Building on this grid, we ask the question underneath it: the anatomical unit a symbol should span--bone, finger, or whole hand. DigitCode answers it by adapting, grouping, and layering HL's alphabet along the hand's unit hierarchy within one code, cutting the symbolic representation's quantization error by three quarters. The lever is the unit, not the quantizer family: at a fixed unit, training-free and learned strong quantizers are interchangeable on reconstruction, while moving down the anatomical hierarchy is what shifts accuracy. The hierarchy also tracks what downstream tasks need. Because a finger is a genuine, enumerable unit, one per-finger token doubles as a training-free, editable handle for jobs a continuous representation cannot address--repairing malformed generated hands, and retargeting them onto robots. We release HandTok, a reproducible testbed, so hand tokenizers can be compared unit-for-unit. Project page: https://digitcode-demo.github.io.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
31 of 46 references.
References · 31
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2017Neural Discrete Representation Learningcited by 4,712
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
- 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
- 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
- 2023Reconstructing Hands in 3D with Transformerscited by 306
- 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
- 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
- 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 206
- 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
19 more
- 2024Image and Video Tokenization with Binary Spherical Quantizationcited by 112
- 2021Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphscited by 110
- 2022VideoDex: Learning Dexterity from Internet Videoscited by 106
- 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
- 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 67
- 2023ContactGen: Generative Contact Modeling for Grasp Generationcited by 53
- 2023HandRefiner: Refining Malformed Hands in Generated Images by Diffusion-based Conditional Inpaintingcited by 42
- 2024MoGenTS: Motion Generation based on Spatial-Temporal Joint Modelingcited by 41
- 2022G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Modelcited by 38
- 2021Contrastive Representation Learning for Hand Shape Estimationcited by 34
- 2025HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Modelscited by 33
- 2024Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generatorcited by 21
- 2024RealisHuman: A Two-Stage Approach for Refining Malformed Human Parts in Generated Imagescited by 10
- 2026ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generationcited by 4
- 2026BEAT: Tokenizing and Generating Symbolic Music by Uniform Temporal Stepscited by 3
- 2025Time-Shifted Token Scheduling for Symbolic Music Generationcited by 2
- 2026M3T: Discrete Multi-Modal Motion Tokens for Sign Language Productioncited by 2
- 2026LaMoGen: Language to Motion Generation Through LLM-Guided Symbolic Inferencecited by 1
- 2026AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modelingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Hand1000: Generating Realistic Hands from Text with Only 1,000 Images
- 2025Text-Driven 3D Hand Motion Generation from Sign Language Data
- 2026HandX: Scaling Bimanual Motion and Interaction Generation
- 2025Multi-Rigid-Body Approximation of Human Hands with Application to Digital Twin ↗
- 2025FingerCap: Fine-grained Finger-level Hand Motion Captioning
- 2025Using Visual Language Models to Control Bionic Hands: Assessment of Object Perception and Grasp Inference
- 2026Morphology and actuation as inductive biases in robotic hand manipulation ↗
- 2026HandVQA: Diagnosing and Improving Fine-Grained Spatial Reasoning about Hands in Vision-Language Models