mlsanityData to 14 Oct 2026

Towards Zero-shot Sign Language Recognition

arXiv 2201.05914 ↗First posted 15 Jan 2022cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

This paper tackles the problem of zero-shot sign language recognition (ZSSLR), where the goal is to leverage models learned over the seen sign classes to recognize the instances of unseen sign classes. In this context, readily available textual sign descriptions and attributes collected from sign language dictionaries are utilized as semantic class representations for knowledge transfer. For this novel problem setup, we introduce three benchmark datasets with their accompanying textual and attribute descriptions to analyze the problem in detail. Our proposed approach builds spatiotemporal models of body and hand regions. By leveraging the descriptive text and attribute embeddings along with these visual representations within a zero-shot learning framework, we show that textual and attribute based class definitions can provide effective knowledge for the recognition of previously unseen sign classes. We additionally introduce techniques to analyze the influence of binary attributes in correct and incorrect zero-shot predictions. We anticipate that the introduced approaches and the accompanying datasets will provide a basis for further exploration of zero-shot learning in sign language recognition.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 33 of its 107 references that are in the map. First posted on arXiv 15 Jan 2022.

Builds on

33 of 107 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  6. 2014Going Deeper with Convolutionscited by 4,241
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  9. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  10. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  11. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  12. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
21 more
  1. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  2. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 255
  3. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 218
  4. 2017Semantic Autoencoder for Zero-Shot Learningcited by 133
  5. 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
  6. 2018MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Languagecited by 118
  7. 2020BSL-1K: Scaling up co-articulated sign language recognition using mouthing cuescited by 106
  8. 2018Video-based Sign Language Recognition without Temporal Segmentationcited by 88
  9. 2015Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptionscited by 64
  10. 2016Less is more: zero-shot learning from online textual documents with noise suppressioncited by 40
  11. 2014Zero Shot Recognition with Unreliable Attributescited by 39
  12. 2020Watch, read and lookup: learning to spot signs from multiple supervisorscited by 35
  13. 2015Transductive Zero-Shot Action Recognition by Word-Vector Embeddingcited by 31
  14. 2019Explaining Models by Propagating Shapley Values of Local Componentscited by 29
  15. 2017Alternative Semantic Representations for Zero-Shot Human Action Recognitioncited by 27
  16. 2015Objects2action: Classifying and localizing actions without any video examplecited by 26
  17. 2015Semantic Embedding Space for Zero-Shot Action Recognitioncited by 19
  18. 2019Action2Vec: A Crossmodal Embedding Approach to Action Learningcited by 19
  19. 2017Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagerycited by 17
  20. 2019Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?cited by 13
  21. 2018Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importancecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 3; 2025: 2; 2026: 4.

13 papers.

Sort:

  1. 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
  2. 2024MASA: Motion-aware Masked Autoencoder with Semantic Alignment for Sign Language Recognitioncited by 13
  3. 2024Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognitioncited by 6
  4. 2023Learnt Contrastive Concept Embeddings for Sign Recognitioncited by 5
  5. 2025Text-Driven 3D Hand Motion Generation from Sign Language Datacited by 5
  6. 2026HandX: Scaling Bimanual Motion and Interaction Generationcited by 2
  7. 2022A Transformer-Based Contrastive Learning Approach for Few-Shot Sign Language Recognitioncited by 1
  8. 2023Learning from What is Already Out There: Few-shot Sign Language Recognition with Online Dictionariescited by 1
  9. 2024SkelCap: Automated Generation of Descriptive Text from Skeleton Keypoint Sequencescited by 0
  10. 2025SSLR: A Semi-Supervised Learning Method for Isolated Sign Language Recognitioncited by 0
  11. 2026Geometry-Aware Metric Learning for Cross-Lingual Few-Shot Sign Language Recognition on Static Hand Keypointscited by 0
  12. 2026Phonological Perception of Sign Language Modelscited by 0
1 more
  1. 2026A Reverse Sign Language Dictionary: Open-Vocabulary Sign Recognition from Continuous Signing via Video Captioning and Description Retrievalcited by 0