mlsanityData to 14 Oct 2026

Direct Translation between Sign Languages

arXiv 2605.20588 ↗First posted 20 May 2026cs.CL cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Sign language translation has made substantial progress between sign and spoken languages, while translation across sign languages remains less explored. Translating directly between sign languages could support communication across signing communities without requiring a shared written language. A cascade of sign-to-text, spoken-language translation, and text-to-sign models offers one route, but can propagate intermediate errors and requires three sequential translation stages. We develop direct sign-to-sign translation, whose training is limited by the scarcity of parallel signing across languages. To address this obstacle, we adapt back-translation to construct cross-lingual pairs from existing text-sign corpora: the source signing is synthesized through a text bridge, while the target remains the gold sign from the original corpus. Using these pairs, we jointly train a single Qwen3-based model for text-to-sign and sign-to-sign translation. The latter generates target signing directly from source signing without an intermediate transcript. Experiments cover six directions among American, Chinese, and German Sign Language, using back-translation pairs and existing cross-lingual sign pairs. On synthetic sources, direct translation improves BLEU-4 in five directions and lowers overall motion error relative to the cascade. On existing test pairs, it improves BLEU-4 over the cascade by 1.91-3.87 points across all six directions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

30 of 44 references.

References · 30

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  9. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  10. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  11. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  12. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
18 more
  1. 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
  2. 2019Direct speech-to-speech translation with a sequence-to-sequence modelcited by 118
  3. 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 75
  4. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  5. 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 71
  6. 2021Including Signed Languages in Natural Language Processingcited by 68
  7. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
  8. 2021Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitivescited by 50
  9. 2023Gloss-Free End-to-End Sign Language Translationcited by 32
  10. 2021Data Augmentation for Sign Language Gloss Translationcited by 28
  11. 2024Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generatorcited by 21
  12. 2022Machine Translation from Signed to Spoken Languages: State of the Art and Challengescited by 20
  13. 2022Machine Translation between Spoken Languages and Signed Languages Represented in SignWritingcited by 18
  14. 2023JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processingcited by 18
  15. 2024Leveraging the Power of MLLMs for Gloss-Free Sign Language Translationcited by 13
  16. 2025Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translationcited by 12
  17. 2025Meaningful Pose-Based Sign Language Evaluationcited by 6
  18. 2024Reconsidering Sentence-Level Sign Language Translationcited by 4

Built on by

Not cited yet.