mlsanityData to 14 Oct 2026

Data Augmentation for Sign Language Gloss Translation

arXiv 2105.07476 ↗First posted 16 May 2021cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

Sign language translation (SLT) is often decomposed into video-to-gloss recognition and gloss-to-text translation, where a gloss is a sequence of transcribed spoken-language words in the order in which they are signed. We focus here on gloss-to-text translation, which we treat as a low-resource neural machine translation (NMT) problem. However, unlike traditional low-resource NMT, gloss-to-text translation differs because gloss-text pairs often have a higher lexical overlap and lower syntactic overlap than pairs of spoken languages. We exploit this lexical overlap and handle syntactic divergence by proposing two rule-based heuristics that generate pseudo-parallel gloss-text pairs from monolingual spoken language text. By pre-training on the thus obtained synthetic data, we improve translation from American Sign Language (ASL) to English and German Sign Language (DGS) to German by up to 3.14 and 2.20 BLEU, respectively.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 14 of its 40 references that are in the map. First posted on arXiv 16 May 2021.

Builds on

14 of 40 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  5. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  6. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  7. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  8. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  9. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 144
  10. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 126
  11. 2019Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspectivecited by 87
  12. 2019Generalized Data Augmentation for Low-Resource Translationcited by 53
2 more
  1. 2020Dynamic Data Selection and Weighting for Iterative Back-Translationcited by 25
  2. 2019Handling Syntactic Divergence in Low-resource Machine Translationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 3; 2024: 9; 2025: 5; 2026: 3.

28 papers.

Sort:

  1. 2022Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Productioncited by 61
  2. 2023SLTUNET: A Simple Unified Model for Sign Language Translationcited by 35
  3. 2022Considerations for meaningful sign language machine translation based on glossescited by 25
  4. 2022Machine Translation from Signed to Spoken Languages: State of the Art and Challengescited by 20
  5. 2022Automatic dense annotation of large-vocabulary sign language videoscited by 20
  6. 2023Cross-modality Data Augmentation for End-to-End Sign Language Translationcited by 16
  7. 2024Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Productioncited by 16
  8. 2022Changing the Representation: Examining Language Representation for Neural Sign Language Productioncited by 15
  9. 2022Modeling Intensification for Sign Language Generation: A Computational Approachcited by 9
  10. 2023An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translationcited by 9
  11. 2024iSign: A Benchmark for Indian Sign Language Processingcited by 6
  12. 2025Meaningful Pose-Based Sign Language Evaluationcited by 6
16 more
  1. 2024Improvement in Sign Language Translation Using Text CTC Alignmentcited by 5
  2. 2022Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translationcited by 4
  3. 2022Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentationcited by 3
  4. 2024Select and Reorder: A Novel Approach for Neural Sign Language Productioncited by 3
  5. 2024Generating Signed Language Instructions in Large-Scale Dialogue Systemscited by 3
  6. 2025Breaking the Silence: A Dataset and Benchmark for Bangla Text-to-Gloss Translationcited by 3
  7. 2025Using Sign Language Production as Data Augmentation to enhance Sign Language Translationcited by 3
  8. 2024SignBLEU: Automatic Evaluation of Multi-channel Sign Language Translationcited by 2
  9. 2024ELMI: Interactive and Intelligent Sign Language Translation of Lyrics for Song Signingcited by 1
  10. 2024Diverse Sign Language Translationcited by 1
  11. 2025POSESTITCH-SLT: Linguistically Inspired Pose-Stitching for End-to-End Sign Language Translationcited by 1
  12. 2024Semi-Supervised Spoken Language Glossificationcited by 0
  13. 2025Co-creation for Sign Language Processing and Machine Translationcited by 0
  14. 2026Direct Translation between Sign Languagescited by 0
  15. 2026Target-Side Paraphrase Augmentation for Sign Language Translation with Large Language Modelscited by 0
  16. 2026Corpus Augmentation for Sign Language Translation via LLM-Guided Video Stitchingcited by 0