# G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Model

arXiv: https://arxiv.org/abs/2208.09141 · HTML: https://mlsanity.com/paper/2208.09141/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Sign language production, one of 14 directions in Diffusion models.
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 19 Aug 2022.

## Placed in

- Diffusion models / [Sign language production](https://mlsanity.com/diffusion/sign-language-production/)

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search](https://mlsanity.com/paper/2005.11129/) (2020)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Generalization in Generation: A closer look at Exposure Bias](https://mlsanity.com/paper/1910.00292/) (2019)
- [Hierarchical Transformers Are More Efficient Language Models](https://mlsanity.com/paper/2110.13711/) (2021)
- [Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer](https://mlsanity.com/paper/2204.08680/) (2022)
- [Multi-channel Transformers for Multi-articulatory Sign Language Translation](https://mlsanity.com/paper/2009.00299/) (2020)
- [Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Production](https://mlsanity.com/paper/2203.15354/) (2022)
- [SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition](https://mlsanity.com/paper/2110.05382/) (2021)
- [Adversarial Training for Multi-Channel Sign Language Production](https://mlsanity.com/paper/2008.12405/) (2020)
- [Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives](https://mlsanity.com/paper/2107.11317/) (2021)
- [PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution](https://mlsanity.com/paper/2107.12600/) (2021)

## Built on by (most cited first, 30 of 38)

- [T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text](https://mlsanity.com/paper/2406.07119/) (2024)
- [Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generator](https://mlsanity.com/paper/2411.17799/) (2024)
- [Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observation](https://mlsanity.com/paper/2411.16810/) (2024)
- [Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production](https://mlsanity.com/paper/2412.13609/) (2024)
- [UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons](https://mlsanity.com/paper/2309.07051/) (2023)
- [Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization](https://mlsanity.com/paper/2506.15980/) (2025)
- [Sign Stitching: A Novel Approach to Sign Language Production](https://mlsanity.com/paper/2405.07663/) (2024)
- [A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Production](https://mlsanity.com/paper/2407.02854/) (2024)
- [Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios](https://mlsanity.com/paper/2410.20359/) (2024)
- [Linguistics-Vision Monotonic Consistent Network for Sign Language Production](https://mlsanity.com/paper/2412.16944/) (2024)
- [Disentangle and Regularize: Sign Language Production with Articulator-Based Disentanglement and Channel-Aware Regularization](https://mlsanity.com/paper/2504.06610/) (2025)
- [MaDiS: Taming Masked Diffusion Language Models for Sign Language Generation](https://mlsanity.com/paper/2601.19577/) (2026)
- [A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation](https://mlsanity.com/paper/2308.08849/) (2023)
- [Text-Driven Diffusion Model for Sign Language Production](https://mlsanity.com/paper/2503.15914/) (2025)
- [SignAligner: Harmonizing Complementary Pose Modalities for Coherent Sign Language Generation](https://mlsanity.com/paper/2506.11621/) (2025)
- [Harmonizing Pixels and Melodies: Maestro-Guided Film Score Generation and Composition Style Transfer](https://mlsanity.com/paper/2411.07539/) (2024)
- [Stable Signer: Hierarchical Sign Language Generative Model](https://mlsanity.com/paper/2512.04048/) (2025)
- [DiffuSIA: A Spiral Interaction Architecture for Encoder-Decoder Text Diffusion](https://mlsanity.com/paper/2305.11517/) (2023)
- [GaitMorph: Transforming Gait by Optimally Transporting Discrete Codes](https://mlsanity.com/paper/2307.14713/) (2023)
- [SIGNER: Temporally Grounded Sign Language Generation via Time-Resolved Conditioning](https://mlsanity.com/paper/2506.07460/) (2025)
- [Motion is the Choreographer: Learning Latent Pose Dynamics for Seamless Sign Language Generation](https://mlsanity.com/paper/2508.04049/) (2025)
- [Text2Sign Diffusion: A Generative Approach for Gloss-Free Sign Language Production](https://mlsanity.com/paper/2509.10845/) (2025)
- [Context-Aware Autoregressive Diffusion for Gloss-Wise Sign Language Production](https://mlsanity.com/paper/2606.21234/) (2026)
- [StgcDiff: Spatial-Temporal Graph Condition Diffusion for Sign Language Transition Generation](https://mlsanity.com/paper/2506.13156/) (2025)
- [Hybrid Autoregressive-Diffusion Model for Real-Time Sign Language Production](https://mlsanity.com/paper/2507.09105/) (2025)
- [Towards Skeletal and Signer Noise Reduction in Sign Language Production via Quaternion-Based Pose Encoding and Contrastive Learning](https://mlsanity.com/paper/2508.14574/) (2025)
- [Geometry-Aware Losses for Structure-Preserving Text-to-Sign Language Generation](https://mlsanity.com/paper/2509.23011/) (2025)
- [A2V-SLP: Alignment-Aware Variational Modeling for Disentangled Sign Language Production](https://mlsanity.com/paper/2602.11861/) (2026)
- [Towards Continuous Sign Language Conversation from Isolated Signs](https://mlsanity.com/paper/2605.14705/) (2026)
- [Conditional Collapse in Sign Language Production: A Diagnostic and a Scaling Argument](https://mlsanity.com/paper/2606.01643/) (2026)

## Abstract (arXiv)

The Sign Language Production (SLP) project aims to automatically translate spoken languages into sign sequences. Our approach focuses on the transformation of sign gloss sequences into their corresponding sign pose sequences (G2P). In this paper, we present a novel solution for this task by converting the continuous pose space generation problem into a discrete sequence generation problem. We introduce the Pose-VQVAE framework, which combines Variational Autoencoders (VAEs) with vector quantization to produce a discrete latent representation for continuous pose sequences. Additionally, we propose the G2P-DDM model, a discrete denoising diffusion architecture for length-varied discrete sequence data, to model the latent prior. To further enhance the quality of pose sequence generation in the discrete space, we present the CodeUnet model to leverage spatial-temporal information. Lastly, we develop a heuristic sequential clustering method to predict variable lengths of pose sequences for corresponding gloss sequences. Our results show that our model outperforms state-of-the-art G2P models on the public SLP evaluation benchmark. For more generated results, please visit our project page: textcolorblueurlhttps://slpdiffusier.github.io/g2p-ddm