# Multilingual Gloss-free Sign Language Translation: Towards Building a Sign Language Foundation Model

arXiv: https://arxiv.org/abs/2505.24355 · HTML: https://mlsanity.com/paper/2505.24355/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Improving Sign Language Translation with Monolingual Data by Sign Back-Translation](https://mlsanity.com/paper/2105.12397/) (2021)
- [Two-Stream Network for Sign Language Recognition and Translation](https://mlsanity.com/paper/2211.01367/) (2022)
- [Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining](https://mlsanity.com/paper/2307.14768/) (2023)
- [LLMs are Good Sign Language Translators](https://mlsanity.com/paper/2404.00925/) (2024)
- [Improving Massively Multilingual ASR With Auxiliary CTC Objectives](https://mlsanity.com/paper/2302.12829/) (2023)
- [Improving Continuous Sign Language Recognition with Cross-Lingual Signs](https://mlsanity.com/paper/2308.10809/) (2023)
- [Revisiting End-to-End Speech-to-Text Translation From Scratch](https://mlsanity.com/paper/2206.04571/) (2022)
- [Considerations for meaningful sign language machine translation based on glosses](https://mlsanity.com/paper/2211.15464/) (2022)
- [Hierarchical Multi Task Learning With CTC](https://mlsanity.com/paper/1807.07104/) (2018)
- [JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processing](https://mlsanity.com/paper/2311.10174/) (2023)
- [Hierarchical Conditional End-to-End ASR with CTC and Multi-Granular Subword Units](https://mlsanity.com/paper/2110.04109/) (2021)
- [FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluation](https://mlsanity.com/paper/2408.13585/) (2024)
- [Improvement in Sign Language Translation Using Text CTC Alignment](https://mlsanity.com/paper/2412.09014/) (2024)

## Built on by (most cited first, 7 of 7)

- [SONAR-SLT: Multilingual Sign Language Translation via Language-Agnostic Sentence Embedding Supervision](https://mlsanity.com/paper/2510.19398/) (2025)
- [A Critical Study of Automatic Evaluation in Sign Language Translation](https://mlsanity.com/paper/2510.25434/) (2025)
- [FEA-SLT: A Gloss-Free End-to-End Framework for Facial-Expression-Aware Sign Language Translation](https://mlsanity.com/paper/2601.03549/) (2026)
- [SLU-2K: A Question-Based Benchmark for Semantic Evaluation of Sign Language Translation](https://mlsanity.com/paper/2606.03788/) (2026)
- [Q-BridgeNet: A Quantization Network for Cross-Lingual Sign Language Translation](https://mlsanity.com/paper/2607.11215/) (2026)
- [DualAnchor: Preserving Language Priors and Improving Lexical Fidelity in Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2607.27614/) (2026)
- [SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation](https://mlsanity.com/paper/2609.21709/) (2026)

## Abstract (arXiv)

Sign Language Translation (SLT) aims to convert sign language (SL) videos into spoken language text, thereby bridging the communication gap between the sign and the spoken community. While most existing works focus on translating a single sign language into a single spoken language (one-to-one SLT), leveraging multilingual resources could mitigate low-resource issues and enhance accessibility. However, multilingual SLT (MLSLT) remains unexplored due to language conflicts and alignment difficulties across SLs and spoken languages. To address these challenges, we propose a multilingual gloss-free model with dual CTC objectives for token-level SL identification and spoken text generation. Our model supports 10 SLs and handles one-to-one, many-to-one, and many-to-many SLT tasks, achieving competitive performance compared to state-of-the-art methods on three widely adopted benchmarks: multilingual SP-10, PHOENIX14T, and CSL-Daily.