# American Sign Language fingerspelling recognition in the wild

arXiv: https://arxiv.org/abs/1810.11438 · HTML: https://mlsanity.com/paper/1810.11438/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2018.

## Built on by (most cited first, 20 of 20)

- [Skeleton Aware Multi-modal Sign Language Recognition](https://mlsanity.com/paper/2103.08833/) (2021)
- [YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpus](https://mlsanity.com/paper/2306.15162/) (2023)
- [Open-Domain Sign Language Translation Learned from Online Video](https://mlsanity.com/paper/2205.12870/) (2022)
- [Including Signed Languages in Natural Language Processing](https://mlsanity.com/paper/2105.05222/) (2021)
- [SignCLIP: Connecting Text and Sign Language by Contrastive Learning](https://mlsanity.com/paper/2407.01264/) (2024)
- [Weakly-supervised Fingerspelling Recognition in British Sign Language Videos](https://mlsanity.com/paper/2211.08954/) (2022)
- [A Fine-Grained Visual Attention Approach for Fingerspelling Recognition in the Wild](https://mlsanity.com/paper/2105.07625/) (2021)
- [Fingerspelling Detection in American Sign Language](https://mlsanity.com/paper/2104.01291/) (2021)
- [Data Representativeness in Accessibility Datasets: A Meta-Analysis](https://mlsanity.com/paper/2207.08037/) (2022)
- [FSboard: Over 3 million characters of ASL fingerspelling collected via smartphones](https://mlsanity.com/paper/2407.15806/) (2024)
- [Searching for fingerspelled content in American Sign Language](https://mlsanity.com/paper/2203.13291/) (2022)
- [Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer Models](https://mlsanity.com/paper/2311.12128/) (2023)
- [Reconsidering Sentence-Level Sign Language Translation](https://mlsanity.com/paper/2406.11049/) (2024)
- [ASL-Homework-RGBD Dataset: An annotated dataset of 45 fluent and non-fluent signers performing American Sign Language homeworks](https://mlsanity.com/paper/2207.04021/) (2022)
- [AirLetters: An Open Video Dataset of Characters Drawn in the Air](https://mlsanity.com/paper/2410.02921/) (2024)
- [HandReader: Advanced Techniques for Efficient Fingerspelling Recognition](https://mlsanity.com/paper/2505.10267/) (2025)
- [SignBind-LLM: Multi-Stage Modality Fusion for Sign Language Translation](https://mlsanity.com/paper/2509.00030/) (2025)
- [An Open-Source American Sign Language Fingerspell Recognition and Semantic Pose Retrieval Interface](https://mlsanity.com/paper/2408.09311/) (2024)
- [OpenFS: Multi-Hand-Capable Fingerspelling Recognition with Implicit Signing-Hand Detection and Frame-Wise Letter-Conditioned Synthesis](https://mlsanity.com/paper/2602.22949/) (2026)
- [Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards](https://mlsanity.com/paper/2606.19352/) (2026)

## Abstract (arXiv)

We address the problem of American Sign Language fingerspelling recognition in the wild, using videos collected from websites. We introduce the largest data set available so far for the problem of fingerspelling recognition, and the first using naturally occurring video data. Using this data set, we present the first attempt to recognize fingerspelling sequences in this challenging setting. Unlike prior work, our video data is extremely challenging due to low frame rates and visual variability. To tackle the visual challenges, we train a special-purpose signing hand detector using a small subset of our data. Given the hand detector output, a sequence model decodes the hypothesized fingerspelled letter sequence. For the sequence model, we explore attention-based recurrent encoder-decoders and CTC-based approaches. As the first attempt at fingerspelling recognition in the wild, this work is intended to serve as a baseline for future work on sign language recognition in realistic conditions. We find that, as expected, letter error rates are much higher than in previous work on more controlled data, and we analyze the sources of error and effects of model variants.