# Data Augmentation for Sign Language Gloss Translation

arXiv: https://arxiv.org/abs/2105.07476 · HTML: https://mlsanity.com/paper/2105.07476/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 16 May 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1604.02201/) (2016)
- [Choosing Transfer Languages for Cross-Lingual Learning](https://mlsanity.com/paper/1905.12688/) (2019)
- [When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?](https://mlsanity.com/paper/1804.06323/) (2018)
- [Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective](https://mlsanity.com/paper/1908.08597/) (2019)
- [Generalized Data Augmentation for Low-Resource Translation](https://mlsanity.com/paper/1906.03785/) (2019)
- [Dynamic Data Selection and Weighting for Iterative Back-Translation](https://mlsanity.com/paper/2004.03672/) (2020)
- [Handling Syntactic Divergence in Low-resource Machine Translation](https://mlsanity.com/paper/1909.00040/) (2019)

## Built on by (most cited first, 28 of 28)

- [Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Production](https://mlsanity.com/paper/2203.15354/) (2022)
- [SLTUNET: A Simple Unified Model for Sign Language Translation](https://mlsanity.com/paper/2305.01778/) (2023)
- [Considerations for meaningful sign language machine translation based on glosses](https://mlsanity.com/paper/2211.15464/) (2022)
- [Machine Translation from Signed to Spoken Languages: State of the Art and Challenges](https://mlsanity.com/paper/2202.03086/) (2022)
- [Automatic dense annotation of large-vocabulary sign language videos](https://mlsanity.com/paper/2208.02802/) (2022)
- [Cross-modality Data Augmentation for End-to-End Sign Language Translation](https://mlsanity.com/paper/2305.11096/) (2023)
- [Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production](https://mlsanity.com/paper/2412.13609/) (2024)
- [Changing the Representation: Examining Language Representation for Neural Sign Language Production](https://mlsanity.com/paper/2210.06312/) (2022)
- [Modeling Intensification for Sign Language Generation: A Computational Approach](https://mlsanity.com/paper/2203.09679/) (2022)
- [An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translation](https://mlsanity.com/paper/2305.17714/) (2023)
- [iSign: A Benchmark for Indian Sign Language Processing](https://mlsanity.com/paper/2407.05404/) (2024)
- [Meaningful Pose-Based Sign Language Evaluation](https://mlsanity.com/paper/2510.07453/) (2025)
- [Improvement in Sign Language Translation Using Text CTC Alignment](https://mlsanity.com/paper/2412.09014/) (2024)
- [Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation](https://mlsanity.com/paper/2210.07054/) (2022)
- [Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation](https://mlsanity.com/paper/2204.05953/) (2022)
- [Select and Reorder: A Novel Approach for Neural Sign Language Production](https://mlsanity.com/paper/2404.11532/) (2024)
- [Generating Signed Language Instructions in Large-Scale Dialogue Systems](https://mlsanity.com/paper/2410.14026/) (2024)
- [Breaking the Silence: A Dataset and Benchmark for Bangla Text-to-Gloss Translation](https://mlsanity.com/paper/2504.02293/) (2025)
- [Using Sign Language Production as Data Augmentation to enhance Sign Language Translation](https://mlsanity.com/paper/2506.09643/) (2025)
- [SignBLEU: Automatic Evaluation of Multi-channel Sign Language Translation](https://mlsanity.com/paper/2406.06648/) (2024)
- [ELMI: Interactive and Intelligent Sign Language Translation of Lyrics for Song Signing](https://mlsanity.com/paper/2409.09760/) (2024)
- [Diverse Sign Language Translation](https://mlsanity.com/paper/2410.19586/) (2024)
- [POSESTITCH-SLT: Linguistically Inspired Pose-Stitching for End-to-End Sign Language Translation](https://mlsanity.com/paper/2511.00270/) (2025)
- [Semi-Supervised Spoken Language Glossification](https://mlsanity.com/paper/2406.08173/) (2024)
- [Co-creation for Sign Language Processing and Machine Translation](https://mlsanity.com/paper/2503.01553/) (2025)
- [Direct Translation between Sign Languages](https://mlsanity.com/paper/2605.20588/) (2026)
- [Target-Side Paraphrase Augmentation for Sign Language Translation with Large Language Models](https://mlsanity.com/paper/2605.31393/) (2026)
- [Corpus Augmentation for Sign Language Translation via LLM-Guided Video Stitching](https://mlsanity.com/paper/2606.11925/) (2026)

## Abstract (arXiv)

Sign language translation (SLT) is often decomposed into video-to-gloss recognition and gloss-to-text translation, where a gloss is a sequence of transcribed spoken-language words in the order in which they are signed. We focus here on gloss-to-text translation, which we treat as a low-resource neural machine translation (NMT) problem. However, unlike traditional low-resource NMT, gloss-to-text translation differs because gloss-text pairs often have a higher lexical overlap and lower syntactic overlap than pairs of spoken languages. We exploit this lexical overlap and handle syntactic divergence by proposing two rule-based heuristics that generate pseudo-parallel gloss-text pairs from monolingual spoken language text. By pre-training on the thus obtained synthetic data, we improve translation from American Sign Language (ASL) to English and German Sign Language (DGS) to German by up to 3.14 and 2.20 BLEU, respectively.