mlsanityData to 14 Oct 2026

Multilingual Gloss-free Sign Language Translation: Towards Building a Sign Language Foundation Model

arXiv 2505.24355 ↗First posted 30 May 2025cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Sign Language Translation (SLT) aims to convert sign language (SL) videos into spoken language text, thereby bridging the communication gap between the sign and the spoken community. While most existing works focus on translating a single sign language into a single spoken language (one-to-one SLT), leveraging multilingual resources could mitigate low-resource issues and enhance accessibility. However, multilingual SLT (MLSLT) remains unexplored due to language conflicts and alignment difficulties across SLs and spoken languages. To address these challenges, we propose a multilingual gloss-free model with dual CTC objectives for token-level SL identification and spoken text generation. Our model supports 10 SLs and handles one-to-one, many-to-one, and many-to-many SLT tasks, achieving competitive performance compared to state-of-the-art methods on three widely adopted benchmarks: multilingual SP-10, PHOENIX14T, and CSL-Daily.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 18 of its 34 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

18 of 34 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  5. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  6. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  7. 2022Two-Stream Network for Sign Language Recognition and Translationcited by 123
  8. 2023Gloss-free Sign Language Translation: Improving from Visual-Language Pretrainingcited by 105
  9. 2024LLMs are Good Sign Language Translatorscited by 77
  10. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  11. 2023Improving Continuous Sign Language Recognition with Cross-Lingual Signscited by 27
  12. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26
6 more
  1. 2022Considerations for meaningful sign language machine translation based on glossescited by 25
  2. 2018Hierarchical Multi Task Learning With CTCcited by 20
  3. 2023JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processingcited by 18
  4. 2021Hierarchical Conditional End-to-End ASR with CTC and Multi-Granular Subword Unitscited by 13
  5. 2024FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluationcited by 10
  6. 2024Improvement in Sign Language Translation Using Text CTC Alignmentcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025SONAR-SLT: Multilingual Sign Language Translation via Language-Agnostic Sentence Embedding Supervisioncited by 3
  2. 2025A Critical Study of Automatic Evaluation in Sign Language Translationcited by 0
  3. 2026FEA-SLT: A Gloss-Free End-to-End Framework for Facial-Expression-Aware Sign Language Translationcited by 0
  4. 2026SLU-2K: A Question-Based Benchmark for Semantic Evaluation of Sign Language Translationcited by 0
  5. 2026Q-BridgeNet: A Quantization Network for Cross-Lingual Sign Language Translationcited by 0
  6. 2026DualAnchor: Preserving Language Priors and Improving Lexical Fidelity in Gloss-Free Sign Language Translationcited by 0
  7. 2026SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generationcited by 0