# Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observation

arXiv: https://arxiv.org/abs/2411.16810 · HTML: https://mlsanity.com/paper/2411.16810/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Sign language production, one of 14 directions in Diffusion models.
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Placed in

- Diffusion models / [Sign language production](https://mlsanity.com/diffusion/sign-language-production/)

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion](https://mlsanity.com/paper/2307.10816/) (2023)
- [DiffIR: Efficient Diffusion Model for Image Restoration](https://mlsanity.com/paper/2303.09472/) (2023)
- [Guided Motion Diffusion for Controllable Human Motion Synthesis](https://mlsanity.com/paper/2305.12577/) (2023)
- [Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison](https://mlsanity.com/paper/1910.11006/) (2019)
- [Conditional Image-to-Video Generation with Latent Flow Diffusion Models](https://mlsanity.com/paper/2303.13744/) (2023)
- [Progressive Transformers for End-to-End Sign Language Production](https://mlsanity.com/paper/2004.14874/) (2020)
- [Diff-Retinex: Rethinking Low-light Image Enhancement with A Generative Diffusion Model](https://mlsanity.com/paper/2308.13164/) (2023)
- [DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation](https://mlsanity.com/paper/2307.01831/) (2023)
- [Video-based Sign Language Recognition without Temporal Segmentation](https://mlsanity.com/paper/1801.10111/) (2018)
- [Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation](https://mlsanity.com/paper/2303.11579/) (2023)
- [LLMs are Good Sign Language Translators](https://mlsanity.com/paper/2404.00925/) (2024)
- [TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translation](https://mlsanity.com/paper/2010.05468/) (2020)
- [Multi-channel Transformers for Multi-articulatory Sign Language Translation](https://mlsanity.com/paper/2009.00299/) (2020)
- [SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding](https://mlsanity.com/paper/2305.04868/) (2023)
- [Quantitative Survey of the State of the Art in Sign Language Recognition](https://mlsanity.com/paper/2008.09918/) (2020)
- [Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives](https://mlsanity.com/paper/2107.11317/) (2021)
- [FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation](https://mlsanity.com/paper/2403.12962/) (2024)
- [G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Model](https://mlsanity.com/paper/2208.09141/) (2022)
- [BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization](https://mlsanity.com/paper/2302.05075/) (2023)
- [DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration](https://mlsanity.com/paper/2303.06682/) (2023)
- [Grid Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2404.00234/) (2024)
- [Including Facial Expressions in Contextual Embeddings for Sign Language Generation](https://mlsanity.com/paper/2202.05383/) (2022)

## Built on by (most cited first, 16 of 16)

- [Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generator](https://mlsanity.com/paper/2411.17799/) (2024)
- [Wi-CBR: Salient-aware Adaptive WiFi Sensing for Cross-domain Behavior Recognition](https://mlsanity.com/paper/2506.11616/) (2025)
- [Towards Fine-Grained Emotion Understanding via Skeleton-Based Micro-Gesture Recognition](https://mlsanity.com/paper/2506.12848/) (2025)
- [SLRTP2025 Sign Language Production Challenge: Methodology, Results, and Future Work](https://mlsanity.com/paper/2508.06951/) (2025)
- [MaDiS: Taming Masked Diffusion Language Models for Sign Language Generation](https://mlsanity.com/paper/2601.19577/) (2026)
- [Text2Lip: Progressive Lip-Synced Talking Face Generation from Text via Viseme-Guided Rendering](https://mlsanity.com/paper/2508.02362/) (2025)
- [Motion is the Choreographer: Learning Latent Pose Dynamics for Seamless Sign Language Generation](https://mlsanity.com/paper/2508.04049/) (2025)
- [Context-Aware Autoregressive Diffusion for Gloss-Wise Sign Language Production](https://mlsanity.com/paper/2606.21234/) (2026)
- [StgcDiff: Spatial-Temporal Graph Condition Diffusion for Sign Language Transition Generation](https://mlsanity.com/paper/2506.13156/) (2025)
- [Hybrid Autoregressive-Diffusion Model for Real-Time Sign Language Production](https://mlsanity.com/paper/2507.09105/) (2025)
- [Spoofing-aware Prompt Learning for Unified Physical-Digital Facial Attack Detection](https://mlsanity.com/paper/2512.06363/) (2025)
- [Towards Continuous Sign Language Conversation from Isolated Signs](https://mlsanity.com/paper/2605.14705/) (2026)
- [Escaping Low-Dimensional Overlap: Multi-Task Model Merging via High-Dimensional Sparse Disentanglement](https://mlsanity.com/paper/2608.25354/) (2026)
- [SignRR: Retrieve and Refine Real Motion for Sign Language Production](https://mlsanity.com/paper/2608.28568/) (2026)
- [SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation](https://mlsanity.com/paper/2609.21709/) (2026)
- [Machine Translation for Sign Languages](https://mlsanity.com/paper/2610.00881/) (2026)

## Abstract (arXiv)

Generating continuous sign language videos from discrete segments is challenging due to the need for smooth transitions that preserve natural flow and meaning. Traditional approaches that simply concatenate isolated signs often result in abrupt transitions, disrupting video coherence. To address this, we propose a novel framework, Sign-D2C, that employs a conditional diffusion model to synthesize contextually smooth transition frames, enabling the seamless construction of continuous sign language sequences. Our approach transforms the unsupervised problem of transition frame generation into a supervised training task by simulating the absence of transition frames through random masking of segments in long-duration sign videos. The model learns to predict these masked frames by denoising Gaussian noise, conditioned on the surrounding sign observations, allowing it to handle complex, unstructured transitions. During inference, we apply a linearly interpolating padding strategy that initializes missing frames through interpolation between boundary frames, providing a stable foundation for iterative refinement by the diffusion model. Extensive experiments on the PHOENIX14T, USTC-CSL100, and USTC-SLR500 datasets demonstrate the effectiveness of our method in producing continuous, natural sign language videos.