# Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representations

arXiv: https://arxiv.org/abs/2309.12179 · HTML: https://mlsanity.com/paper/2309.12179/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity](https://mlsanity.com/paper/2009.02119/) (2020)
- [How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language](https://mlsanity.com/paper/2008.08143/) (2020)
- [Progressive Transformers for End-to-End Sign Language Production](https://mlsanity.com/paper/2004.14874/) (2020)
- [Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots](https://mlsanity.com/paper/1810.12541/) (2018)
- [A Neural Temporal Model for Human Motion Prediction](https://mlsanity.com/paper/1809.03036/) (2018)
- [Open-Domain Sign Language Translation Learned from Online Video](https://mlsanity.com/paper/2205.12870/) (2022)
- [Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Production](https://mlsanity.com/paper/2203.15354/) (2022)
- [Adversarial Training for Multi-Channel Sign Language Production](https://mlsanity.com/paper/2008.12405/) (2020)
- [Gloss Attention for Gloss-free Sign Language Translation](https://mlsanity.com/paper/2307.07361/) (2023)
- [Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives](https://mlsanity.com/paper/2107.11317/) (2021)
- [CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning](https://mlsanity.com/paper/2303.12793/) (2023)
- [Gloss-Free End-to-End Sign Language Translation](https://mlsanity.com/paper/2305.12876/) (2023)
- [Considerations for meaningful sign language machine translation based on glosses](https://mlsanity.com/paper/2211.15464/) (2022)
- [Sign Language Video Retrieval with Free-Form Textual Queries](https://mlsanity.com/paper/2201.02495/) (2022)
- [Ham2Pose: Animating Sign Language Notation into Pose Sequences](https://mlsanity.com/paper/2211.13613/) (2022)
- [Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker](https://mlsanity.com/paper/2305.13729/) (2023)
- [Unsupervised Document Expansion for Information Retrieval with Stochastic Text Generation](https://mlsanity.com/paper/2105.00666/) (2021)

## Built on by (most cited first, 9 of 9)

- [SignCLIP: Connecting Text and Sign Language by Contrastive Learning](https://mlsanity.com/paper/2407.01264/) (2024)
- [A Data-Driven Representation for Sign Language Production](https://mlsanity.com/paper/2404.11499/) (2024)
- [Sign Stitching: A Novel Approach to Sign Language Production](https://mlsanity.com/paper/2405.07663/) (2024)
- [Meaningful Pose-Based Sign Language Evaluation](https://mlsanity.com/paper/2510.07453/) (2025)
- [Disentangle and Regularize: Sign Language Production with Articulator-Based Disentanglement and Channel-Aware Regularization](https://mlsanity.com/paper/2504.06610/) (2025)
- [SLRTP2025 Sign Language Production Challenge: Methodology, Results, and Future Work](https://mlsanity.com/paper/2508.06951/) (2025)
- [Studying and Mitigating Biases in Sign Language Understanding Models](https://mlsanity.com/paper/2410.05206/) (2024)
- [Teach Me Sign: Stepwise Prompting LLM for Sign Language Production](https://mlsanity.com/paper/2507.10972/) (2025)
- [A2V-SLP: Alignment-Aware Variational Modeling for Disentangled Sign Language Production](https://mlsanity.com/paper/2602.11861/) (2026)

## Abstract (arXiv)

Gloss-free Sign Language Production (SLP) offers a direct translation of spoken language sentences into sign language, bypassing the need for gloss intermediaries. This paper presents the Sign language Vector Quantization Network, a novel approach to SLP that leverages Vector Quantization to derive discrete representations from sign pose sequences. Our method, rooted in both manual and non-manual elements of signing, supports advanced decoding methods and integrates latent-level alignment for enhanced linguistic coherence. Through comprehensive evaluations, we demonstrate superior performance of our method over prior SLP methods and highlight the reliability of Back-Translation and Fr'echet Gesture Distance as evaluation metrics.