# Text2Sign Diffusion: A Generative Approach for Gloss-Free Sign Language Production

arXiv: https://arxiv.org/abs/2509.10845 · HTML: https://mlsanity.com/paper/2509.10845/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Sign language production, one of 14 directions in Diffusion models.
- Builds on 31 of its 48 references that are in the map.
- First posted on arXiv 13 Sep 2025.

## Placed in

- Diffusion models / [Sign language production](https://mlsanity.com/diffusion/sign-language-production/)

## Builds on (most cited first, 30 of 31)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language](https://mlsanity.com/paper/2008.08143/) (2020)
- [Two-Stream Network for Sign Language Recognition and Translation](https://mlsanity.com/paper/2211.01367/) (2022)
- [Progressive Transformers for End-to-End Sign Language Production](https://mlsanity.com/paper/2004.14874/) (2020)
- [Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining](https://mlsanity.com/paper/2307.14768/) (2023)
- [A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation](https://mlsanity.com/paper/2203.04287/) (2022)
- [LLMs are Good Sign Language Translators](https://mlsanity.com/paper/2404.00925/) (2024)
- [Neural Sign Language Translation based on Human Keypoint Estimation](https://mlsanity.com/paper/1811.11436/) (2018)
- [Gloss Attention for Gloss-free Sign Language Translation](https://mlsanity.com/paper/2307.07361/) (2023)
- [Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks](https://mlsanity.com/paper/2103.06982/) (2021)
- [Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives](https://mlsanity.com/paper/2107.11317/) (2021)
- [Neural Sign Actors: A diffusion model for 3D sign language production from text](https://mlsanity.com/paper/2312.02702/) (2023)
- [G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Model](https://mlsanity.com/paper/2208.09141/) (2022)
- [Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generation](https://mlsanity.com/paper/2309.03467/) (2023)
- [Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks](https://mlsanity.com/paper/2101.10642/) (2021)
- [Gloss-Free End-to-End Sign Language Translation](https://mlsanity.com/paper/2305.12876/) (2023)
- [Sign Language Translation with Hierarchical Spatio-TemporalGraph Neural Network](https://mlsanity.com/paper/2111.07258/) (2021)
- [Ham2Pose: Animating Sign Language Notation into Pose Sequences](https://mlsanity.com/paper/2211.13613/) (2022)
- [A Data-Driven Representation for Sign Language Production](https://mlsanity.com/paper/2404.11499/) (2024)
- [RI-MAE: Rotation-Invariant Masked AutoEncoders for Self-Supervised Point Cloud Representation Learning](https://mlsanity.com/paper/2409.00353/) (2024)
- [Sign Language Production with Latent Motion Transformer](https://mlsanity.com/paper/2312.12917/) (2023)
- [Including Facial Expressions in Contextual Embeddings for Sign Language Generation](https://mlsanity.com/paper/2202.05383/) (2022)
- [LocSelect: Target Speaker Localization with an Auditory Selective Hearing Mechanism](https://mlsanity.com/paper/2310.10497/) (2023)
- [Multi-Scale Control Signal-Aware Transformer for Motion Synthesis without Phase](https://mlsanity.com/paper/2303.01685/) (2023)
- [MS2SL: Multimodal Spoken Data-Driven Continuous Sign Language Production](https://mlsanity.com/paper/2407.12842/) (2024)

## Built on by (most cited first, 1 of 1)

- [The Impact of VAE Design on Latent Pose Representations for Diffusion-based Sign Language Production](https://mlsanity.com/paper/2606.22959/) (2026)

## Abstract (arXiv)

Sign language production (SLP) aims to translate spoken language sentences into a sequence of pose frames in a sign language, bridging the communication gap and promoting digital inclusion for deaf and hard-of-hearing communities. Existing methods typically rely on gloss, a symbolic representation of sign language words or phrases that serves as an intermediate step in SLP. This limits the flexibility and generalization of SLP, as gloss annotations are often unavailable and language-specific. Therefore, we present a novel diffusion-based generative approach - Text2Sign Diffusion (Text2SignDiff) for gloss-free SLP. Specifically, a gloss-free latent diffusion model is proposed to generate sign language sequences from noisy latent sign codes and spoken text jointly, reducing the potential error accumulation through a non-autoregressive iterative denoising process. We also design a cross-modal signing aligner that learns a shared latent space to bridge visual and textual content in sign and spoken languages. This alignment supports the conditioned diffusion-based process, enabling more accurate and contextually relevant sign language generation without gloss. Extensive experiments on the commonly used PHOENIX14T and How2Sign datasets demonstrate the effectiveness of our method, achieving the state-of-the-art performance.