Machine Translation for Sign Languages
Not on the map. Why?
Abstract (arXiv)
Sign language machine translation has progressed substantially over the past decade, evolving from isolated sign recognition to end-to-end translation systems. Advances in pose estimation, transformer architectures, and large-scale dataset collection have driven progress, yet challenges remain. Datasets are limited compared to spoken-language resources; evaluation metrics inadequately capture the linguistic quality of output; and models must capture the simultaneous, multi-layered, and three-dimensional structure of sign languages. This manuscript provides a comprehensive review that seeks to balance technical challenges with stakeholder considerations. We examine the linguistic properties that make sign languages computationally unique, trace the evolution of recognition, translation, and production systems, and analyze ongoing technical challenges. Crucially, we address ethical considerations around data governance, community involvement, and appropriate use. Drawing on interdisciplinary perspectives spanning computer vision, sign language linguistics, and deaf studies, our analysis emphasizes that continued progress requires sustained collaboration across these fields and with deaf communities.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 51 of its 118 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
51 of 118 references.
References · 51
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
- 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
- 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
- 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 218
- 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
- 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
- 2022Two-Stream Network for Sign Language Recognition and Translationcited by 123
- 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
- 2018MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Languagecited by 118
- 2020BSL-1K: Scaling up co-articulated sign language recognition using mouthing cuescited by 106
- 2023Gloss-free Sign Language Translation: Improving from Visual-Language Pretrainingcited by 105
38 more
- 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
- 2019Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspectivecited by 87
- 2024LLMs are Good Sign Language Translatorscited by 77
- 2020AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methodscited by 75
- 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 75
- 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
- 2018Neural Sign Language Translation based on Human Keypoint Estimationcited by 65
- 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
- 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
- 2021Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitivescited by 50
- 2022G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Modelcited by 38
- 2024YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpuscited by 36
- 2024T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Textcited by 30
- 2019Fingerspelling recognition in the wild with iterative visual attentioncited by 28
- 2021SimulSLT: End-to-End Simultaneous Sign Language Translationcited by 27
- 2021Sign Language Production: A Reviewcited by 26
- 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 26
- 2022Considerations for meaningful sign language machine translation based on glossescited by 25
- 2023SignDiff: Diffusion Model for American Sign Language Productioncited by 25
- 2024C2RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrievalcited by 24
- 2024SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Predictioncited by 22
- 2024Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generatorcited by 21
- 2024SignCLIP: Connecting Text and Sign Language by Contrastive Learningcited by 20
- 2020BosphorusSign22k Sign Language Recognition Datasetcited by 19
- 2024Systemic Biases in Sign Language AI Research: A Deaf-Led Call to Reevaluate Research Agendascited by 19
- 2024Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observationcited by 16
- 2023Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discoursecited by 15
- 2024A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervisioncited by 15
- 2025SignRep: Enhancing Self-Supervised Sign Representationscited by 15
- 2020Sign language segmentation with temporal convolutional networkscited by 14
- 2023Sign Language Production with Latent Motion Transformercited by 13
- 2025Towards AI-driven Sign Language Generation with Non-manual Markerscited by 13
- 2022Weakly-supervised Fingerspelling Recognition in British Sign Language Videoscited by 10
- 2025Hands-On: Segmenting Individual Signs from Continuous Sequencescited by 10
- 2025Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenizationcited by 9
- 2026Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Fieldcited by 9
- 2024Pose-Guided Fine-Grained Sign Language Video Generationcited by 8
- 2025Deaf in AI: AI language technologies and the erosion of linguistic rightscited by 2
50 most cited of 51 All 51 (JSON)
Built on by
Not cited yet.