Direct Translation between Sign Languages
Not on the map. Why?
Abstract (arXiv)
Sign language translation has made substantial progress between sign and spoken languages, while translation across sign languages remains less explored. Translating directly between sign languages could support communication across signing communities without requiring a shared written language. A cascade of sign-to-text, spoken-language translation, and text-to-sign models offers one route, but can propagate intermediate errors and requires three sequential translation stages. We develop direct sign-to-sign translation, whose training is limited by the scarcity of parallel signing across languages. To address this obstacle, we adapt back-translation to construct cross-lingual pairs from existing text-sign corpora: the source signing is synthesized through a text bridge, while the target remains the gold sign from the original corpus. Using these pairs, we jointly train a single Qwen3-based model for text-to-sign and sign-to-sign translation. The latter generates target signing directly from source signing without an intermediate transcript. Experiments cover six directions among American, Chinese, and German Sign Language, using back-translation pairs and existing cross-lingual sign pairs. On synthetic sources, direct translation improves BLEU-4 in five directions and lowers overall motion error relative to the cascade. On existing test pairs, it improves BLEU-4 over the cascade by 1.91-3.87 points across all six directions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
30 of 44 references.
References · 30
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2025Qwen3 Technical Reportcited by 8,820
- 2017Neural Discrete Representation Learningcited by 4,712
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
- 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
- 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
- 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
- 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
- 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
18 more
- 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
- 2019Direct speech-to-speech translation with a sequence-to-sequence modelcited by 118
- 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 75
- 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
- 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 71
- 2021Including Signed Languages in Natural Language Processingcited by 68
- 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
- 2021Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitivescited by 50
- 2023Gloss-Free End-to-End Sign Language Translationcited by 32
- 2021Data Augmentation for Sign Language Gloss Translationcited by 28
- 2024Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generatorcited by 21
- 2022Machine Translation from Signed to Spoken Languages: State of the Art and Challengescited by 20
- 2022Machine Translation between Spoken Languages and Signed Languages Represented in SignWritingcited by 18
- 2023JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processingcited by 18
- 2024Leveraging the Power of MLLMs for Gloss-Free Sign Language Translationcited by 13
- 2025Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translationcited by 12
- 2025Meaningful Pose-Based Sign Language Evaluationcited by 6
- 2024Reconsidering Sentence-Level Sign Language Translationcited by 4
Built on by
Not cited yet.