mlsanityData to 14 Oct 2026

An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translation

arXiv 2305.17714 ↗First posted 28 May 2023cs.CL cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Sign language translation systems are complex and require many components. As a result, it is very hard to compare methods across publications. We present an open-source implementation of a text-to-gloss-to-pose-to-video pipeline approach, demonstrating conversion from German to Swiss German Sign Language, French to French Sign Language of Switzerland, and Italian to Italian Sign Language of Switzerland. We propose three different components for the text-to-gloss translation: a lemmatizer, a rule-based word reordering and dropping component, and a neural machine translation system. Gloss-to-pose conversion occurs using data from a lexicon for three different signed languages, with skeletal poses extracted from videos. To generate a sentence, the text-to-gloss system is first run, and the pose representations of the resulting signs are stitched together.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 18 of its 77 references that are in the map. First posted on arXiv 28 May 2023.

Builds on

18 of 77 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  5. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  6. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  7. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  8. 2018Video-to-Video Synthesiscited by 390
  9. 2018Everybody Dance Nowcited by 315
  10. 2016Using the Output Embedding to Improve Language Modelscited by 272
  11. 2019Revisiting Low-Resource Neural Machine Translation: A Case Studycited by 61
  12. 2020Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Videocited by 34
6 more
  1. 2021Data Augmentation for Sign Language Gloss Translationcited by 28
  2. 2022Considerations for meaningful sign language machine translation based on glossescited by 25
  3. 2022Ham2Pose: Animating Sign Language Notation into Pose Sequencescited by 21
  4. 2021AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video Anonymisationcited by 16
  5. 2020Can Everybody Sign Now? Exploring Sign Language Video Generation from 2D Posescited by 12
  6. 2022Sockeye 3: Fast Neural Machine Translation with PyTorchcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 4; 2026: 1.

9 papers.

  1. 2025Meaningful Pose-Based Sign Language Evaluationcited by 6
  2. 2025Explanatory Summarization with Discourse-Driven Planningcited by 4
  3. 2024Pose-Based Sign Language Appearance Transfercited by 2
  4. 2024Diversity-Aware Sign Language Production through a Pose Encoding Variational Autoencodercited by 1
  5. 2025MATE: LLM-Powered Multi-Agent Translation Environment for Accessibility Applicationscited by 1
  6. 2025Introducing A Bangla Sentence - Gloss Pair Dataset for Bangla Sign Language Translation and Researchcited by 1
  7. 2023sign.mt: Real-Time Multilingual Sign Language Translation Applicationcited by 0
  8. 2024An Open-Source American Sign Language Fingerspell Recognition and Semantic Pose Retrieval Interfacecited by 0
  9. 2026Evaluation of Pose Estimation Systems for Sign Language Translationcited by 0