mlsanityData to 14 Oct 2026

Open-Domain Sign Language Translation Learned from Online Video

arXiv 2205.12870 ↗First posted 25 May 2022cs.CV cs.CLcited by 72

Not on the map. Why?

Abstract (arXiv)

Existing work on sign language translation - that is, translation from sign language videos into sentences in a written language - has focused mainly on (1) data collected in a controlled environment or (2) data in a specific domain, which limits the applicability to real-world settings. In this paper, we introduce OpenASL, a large-scale American Sign Language (ASL) - English dataset collected from online video sites (e.g., YouTube). OpenASL contains 288 hours of ASL videos in multiple domains from over 200 signers and is the largest publicly available ASL translation dataset to date. To tackle the challenges of sign language translation in realistic settings and without glosses, we propose a set of techniques including sign search as a pretext task for pre-training and fusion of mouthing and handshape features. The proposed techniques produce consistent and large improvements in translation quality, over baseline models based on prior work. Our data and code are publicly available at https://github.com/chevalierNoir/OpenASL

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). Builds on 27 of its 45 references that are in the map. First posted on arXiv 25 May 2022.

Builds on

27 of 45 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  4. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  6. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 287
  7. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  8. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 218
  9. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  10. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
  11. 2020BSL-1K: Scaling up co-articulated sign language recognition using mouthing cuescited by 106
  12. 2019Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspectivecited by 87
15 more
  1. 2022A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translationcited by 80
  2. 2020TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translationcited by 75
  3. 2020Multi-channel Transformers for Multi-articulatory Sign Language Translationcited by 71
  4. 2021Including Signed Languages in Natural Language Processingcited by 68
  5. 2018Neural Sign Language Translation based on Human Keypoint Estimationcited by 65
  6. 2020Watch, read and lookup: learning to spot signs from multiple supervisorscited by 35
  7. 2021Read and Attend: Temporal Localisation in Sign Language Videoscited by 34
  8. 2021Content4All Open Research Sign Language Translation Datasetscited by 30
  9. 2019Fingerspelling recognition in the wild with iterative visual attentioncited by 28
  10. 2021BBC-Oxford British Sign Language Datasetcited by 28
  11. 2021Aligning Subtitles in Sign Language Videoscited by 26
  12. 2018American Sign Language fingerspelling recognition in the wildcited by 20
  13. 2020Neural Sign Language Translation by Learning Tokenizationcited by 19
  14. 2021Fingerspelling Detection in American Sign Languagecited by 7
  15. 2022Searching for fingerspelled content in American Sign Languagecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 11; 2024: 26; 2025: 14; 2026: 19.

72 papers.

Sort:

By area:

  1. 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 75
  2. 2024Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translationcited by 57
  3. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 56
  4. 2023Natural Language-Assisted Sign Language Recognitioncited by 46
  5. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 44
  6. 2023Sign Language Translation from Instructional Videoscited by 39
  7. 2024YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpuscited by 36
  8. 2023Gloss-Free End-to-End Sign Language Translationcited by 32
  9. 2024T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Textcited by 30
  10. 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 26
  11. 2025Human Motion Video Generation: A Surveycited by 25
  12. 2024Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translationcited by 24
38 more
  1. 2024C2RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrievalcited by 24
  2. 2024SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Predictioncited by 22
  3. 2024Scaling up Multimodal Pre-training for Sign Language Understandingcited by 21
  4. 2024Systemic Biases in Sign Language AI Research: A Deaf-Led Call to Reevaluate Research Agendascited by 19
  5. 2024SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scalecited by 19
  6. 2023JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processingcited by 18
  7. 2024A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervisioncited by 15
  8. 2025Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translationcited by 12
  9. 2024FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluationcited by 10
  10. 2023Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representationscited by 9
  11. 2024An Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMscited by 9
  12. 2026Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Fieldcited by 9
  13. 2024Scaling Sign Language Translationcited by 8
  14. 2024ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articlescited by 7
  15. 2023Self-Supervised Video Transformers for Isolated Sign Language Recognitioncited by 6
  16. 2023A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Datasetcited by 6
  17. 2024iSign: A Benchmark for Indian Sign Language Processingcited by 6
  18. 2024FSboard: Over 3 million characters of ASL fingerspelling collected via smartphonescited by 6
  19. 2025Isharah: A Large-Scale Multi-Scene Dataset for Continuous Sign Language Recognitioncited by 6
  20. 2023Towards the extraction of robust sign embeddings for low resource sign language recognitioncited by 5
  21. 2023Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer Modelscited by 5
  22. 2024A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Productioncited by 5
  23. 2024Unsupervised Sign Language Translation and Generationcited by 4
  24. 2024Reconsidering Sentence-Level Sign Language Translationcited by 4
  25. 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Spacecited by 4
  26. 2025EmoSign: A Multimodal Dataset for Understanding Emotions in American Sign Languagecited by 4
  27. 2022Clean Text and Full-Body Transformer: Microsoft's Submission to the WMT22 Shared Task on Sign Language Translationcited by 3
  28. 20242M-BELEBELE: Highly Multilingual Speech and American Sign Language Comprehension Datasetcited by 3
  29. 2023ISLTranslate: Dataset for Translating Indian Sign Languagecited by 2
  30. 2024SignLLM: Sign Language Production Large Language Modelscited by 2
  31. 2025Exploring Pose-based Sign Language Translation: Ablation Studies and Attention Insightscited by 2
  32. 2025RVLF: A Reinforcing Vision-Language Framework for Gloss-Free Sign Language Translationcited by 2
  33. 2022Topic Detection in Continuous Sign Language Videoscited by 1
  34. 2024AirLetters: An Open Video Dataset of Characters Drawn in the Aircited by 1
  35. 2024Diverse Sign Language Translationcited by 1
  36. 2024AzSLD: Azerbaijani Sign Language Dataset for Fingerspelling, Word, and Sentence Translation with Baseline Softwarecited by 1
  37. 2025Sign Spotting Disambiguation using Large Language Modelscited by 1
  38. 2025Contrastive Pretraining with Dual Visual Encoders for Gloss-Free Sign Language Translationcited by 1

50 most cited of 72 All 72 (JSON)