mlsanityData to 7 Oct 2026

Sign Language Translation from Instructional Videos

arXiv 2304.06371 ↗First posted 13 Apr 2023cs.CL cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

The advances in automatic sign language translation (SLT) to spoken languages have been mostly benchmarked with datasets of limited size and restricted domains. Our work advances the state of the art by providing the first baseline results on How2Sign, a large and broad dataset. We train a Transformer over I3D video features, using the reduced BLEU as a reference metric for validation, instead of the widely used BLEU score. We report a result of 8.03 on the BLEU score, and publish the first open-source implementation of its kind to promote further advances.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 42 of its 62 references that are in the map. First posted on arXiv 13 Apr 2023.

Builds on

42 of 62 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2017The Kinetics Human Action Video Datasetcited by 1,894
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  9. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  10. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  11. 2016Convolutional Pose Machinescited by 325
  12. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 254
30 more
  1. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 217
  2. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
  3. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 172
  4. 2022Two-Stream Network for Sign Language Recognition and Translationcited by 122
  5. 2018MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Languagecited by 118
  6. 2020BSL-1K: Scaling up co-articulated sign language recognition using mouthing cuescited by 106
  7. 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
  8. 2022A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translationcited by 80
  9. 2021Visual Alignment Constraint for Continuous Sign Language Recognitioncited by 79
  10. 2020AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methodscited by 75
  11. 2020TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translationcited by 75
  12. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  13. 2020Multi-channel Transformers for Multi-articulatory Sign Language Translationcited by 71
  14. 2021Including Signed Languages in Natural Language Processingcited by 67
  15. 2018Neural Sign Language Translation based on Human Keypoint Estimationcited by 65
  16. 2022Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Productioncited by 61
  17. 2021Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networkscited by 50
  18. 2021Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitivescited by 50
  19. 2021Stochastic Transformer Networks with Linear Competing Units: Application to end-to-end SL Translationcited by 35
  20. 2020Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Videocited by 34
  21. 2021Content4All Open Research Sign Language Translation Datasetscited by 30
  22. 2020Real-Time Sign Language Detection using Human Pose Estimationcited by 28
  23. 2020Optimizing Transformer for Low-Resource Neural Machine Translationcited by 24
  24. 2022Sign Language Video Retrieval with Free-Form Textual Queriescited by 21
  25. 2020Neural Sign Language Translation by Learning Tokenizationcited by 19
  26. 2021Skeletal Graph Self-Attention: Embedding a Skeleton Inductive Bias into Sign Language Productioncited by 11
  27. 2022Keypoint based Sign Language Translation without Glossescited by 4
  28. 2022Clean Text and Full-Body Transformer: Microsoft's Submission to the WMT22 Shared Task on Sign Language Translationcited by 3
  29. 2022Tackling Low-Resourced Sign Language Translation: UPC at WMT-SLT 22cited by 3
  30. 2022Topic Detection in Continuous Sign Language Videoscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 10; 2025: 18; 2026: 9.

39 papers.

Sort:

By area:

  1. 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 74
  2. 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 55
  3. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 42
  4. 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 25
  5. 2024Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translationcited by 23
  6. 2024C2RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrievalcited by 23
  7. 2023JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processingcited by 18
  8. 2024A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatarscited by 18
  9. 2024SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scalecited by 18
  10. 2025SignRep: Enhancing Self-Supervised Sign Representationscited by 15
  11. 2025Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translationcited by 12
  12. 2025Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translationcited by 11
27 more
  1. 2024An Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMscited by 9
  2. 2024FSboard: Over 3 million characters of ASL fingerspelling collected via smartphonescited by 6
  3. 2025Sign Language Translation with Sentence Embedding Supervisioncited by 6
  4. 2025Lost in Translation, Found in Embeddings: Sign Language Translation and Alignmentcited by 6
  5. 2024Reconsidering Sentence-Level Sign Language Translationcited by 4
  6. 2025Understanding Co-speech Gestures in-the-wildcited by 3
  7. 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Spacecited by 3
  8. 2024SignLLM: Sign Language Production Large Language Modelscited by 2
  9. 2025Stable Signer: Hierarchical Sign Language Generative Modelcited by 2
  10. 2025RVLF: A Reinforcing Vision-Language Framework for Gloss-Free Sign Language Translationcited by 2
  11. 2025SignBind-LLM: Multi-Stage Modality Fusion for Sign Language Translationcited by 1
  12. 2025MultimodalHugs: Enabling Sign Language Processing in Hugging Facecited by 1
  13. 2025SignMouth: Leveraging Mouthing Cues for Sign Language Translation by Multimodal Contrastive Fusioncited by 1
  14. 2026Think in Latent Thoughts: A New Paradigm for Gloss-Free Sign Language Translationcited by 1
  15. 2026SIGNET: Motion-Level Knowledge Transfer for Cross-Language Sign Language Translationcited by 1
  16. 2024American Sign Language Video to Text Translationcited by 0
  17. 2025Graph Network for Sign Language Taskscited by 0
  18. 2025iLSU-T: an Open Dataset for Uruguayan Sign Language Translationcited by 0
  19. 2025Seeing, Signing, and Saying: A Vision-Language Model-Assisted Pipeline for Sign Language Data Acquisition and Curation from Social Mediacited by 0
  20. 2025Sign language recognition from skeletal data using graph and recurrent neural networkscited by 0
  21. 2026The Influence of Iconicity in Transfer Learning for Sign Language Recognitioncited by 0
  22. 2026Evaluation of Pose Estimation Systems for Sign Language Translationcited by 0
  23. 2026SignVerse-2M: A Two-Million-Clip Pose-Native Universe of 55+ Sign Languagescited by 0
  24. 2026Conditional Collapse in Sign Language Production: A Diagnostic and a Scaling Argumentcited by 0
  25. 2026Attention-Steered Vision-Language Models for Sign Language Translationcited by 0
  26. 2026SignMimic: Robust High-Quality Sign Language Motion Generation via Human-Shape-Oblivious Pose Transfer Guidancecited by 0
  27. 2026OneSign: Unifying Sign Language Understanding Tasks with One Modelcited by 0