# Zero-Shot Sign Language Recognition: Can Textual Data Uncover Sign Languages?

arXiv: https://arxiv.org/abs/1907.10292 · HTML: https://mlsanity.com/paper/1907.10292/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2019.

## Built on by (most cited first, 13 of 13)

- [Towards Zero-shot Sign Language Recognition](https://mlsanity.com/paper/2201.05914/) (2022)
- [Scaling up sign spotting through sign language dictionaries](https://mlsanity.com/paper/2205.04152/) (2022)
- [ASL Video Corpora & Sign Bank: Resources Available through the American Sign Language Linguistic Research Project (ASLLRP)](https://mlsanity.com/paper/2201.07899/) (2022)
- [Looking for the Signs: Identifying Isolated Sign Instances in Continuous Video Footage](https://mlsanity.com/paper/2108.04229/) (2021)
- [Text-Driven 3D Hand Motion Generation from Sign Language Data](https://mlsanity.com/paper/2508.15902/) (2025)
- [ZS-SLR: Zero-Shot Sign Language Recognition from RGB-D Videos](https://mlsanity.com/paper/2108.10059/) (2021)
- [Multi-Modal Zero-Shot Sign Language Recognition](https://mlsanity.com/paper/2109.00796/) (2021)
- [Fine-tuning of sign language recognition models: a technical report](https://mlsanity.com/paper/2302.07693/) (2023)
- [HandX: Scaling Bimanual Motion and Interaction Generation](https://mlsanity.com/paper/2603.28766/) (2026)
- [Learning from What is Already Out There: Few-shot Sign Language Recognition with Online Dictionaries](https://mlsanity.com/paper/2301.03769/) (2023)
- [Transfer: Cross Modality Knowledge Transfer using Adversarial Networks -- A Study on Gesture Recognition](https://mlsanity.com/paper/2306.15114/) (2023)
- [SkelCap: Automated Generation of Descriptive Text from Skeleton Keypoint Sequences](https://mlsanity.com/paper/2405.02977/) (2024)
- [Representing Signs as Signs: One-Shot ISLR to Facilitate Functional Sign Language Technologies](https://mlsanity.com/paper/2502.20171/) (2025)

## Abstract (arXiv)

We introduce the problem of zero-shot sign language recognition (ZSSLR), where the goal is to leverage models learned over the seen sign class examples to recognize the instances of unseen signs. To this end, we propose to utilize the readily available descriptions in sign language dictionaries as an intermediate-level semantic representation for knowledge transfer. We introduce a new benchmark dataset called ASL-Text that consists of 250 sign language classes and their accompanying textual descriptions. Compared to the ZSL datasets in other domains (such as object recognition), our dataset consists of limited number of training examples for a large number of classes, which imposes a significant challenge. We propose a framework that operates over the body and hand regions by means of 3D-CNNs, and models longer temporal relationships via bidirectional LSTMs. By leveraging the descriptive text embeddings along with these spatio-temporal representations within a zero-shot learning framework, we show that textual data can indeed be useful in uncovering sign languages. We anticipate that the introduced approach and the accompanying dataset will provide a basis for further exploration of this new zero-shot learning problem.