# Lexicon-Free Fingerspelling Recognition from Video: Data, Models, and Signer Adaptation

arXiv: https://arxiv.org/abs/1609.07876 · HTML: https://mlsanity.com/paper/1609.07876/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2016.

## Built on by (most cited first, 6 of 6)

- [Weakly-supervised Fingerspelling Recognition in British Sign Language Videos](https://mlsanity.com/paper/2211.08954/) (2022)
- [A Fine-Grained Visual Attention Approach for Fingerspelling Recognition in the Wild](https://mlsanity.com/paper/2105.07625/) (2021)
- [Fingerspelling Detection in American Sign Language](https://mlsanity.com/paper/2104.01291/) (2021)
- [FSboard: Over 3 million characters of ASL fingerspelling collected via smartphones](https://mlsanity.com/paper/2407.15806/) (2024)
- [Searching for fingerspelled content in American Sign Language](https://mlsanity.com/paper/2203.13291/) (2022)
- [Bukva: Russian Sign Language Alphabet](https://mlsanity.com/paper/2410.08675/) (2024)

## Abstract (arXiv)

We study the problem of recognizing video sequences of fingerspelled letters in American Sign Language (ASL). Fingerspelling comprises a significant but relatively understudied part of ASL. Recognizing fingerspelling is challenging for a number of reasons: It involves quick, small motions that are often highly coarticulated; it exhibits significant variation between signers; and there has been a dearth of continuous fingerspelling data collected. In this work we collect and annotate a new data set of continuous fingerspelling videos, compare several types of recognizers, and explore the problem of signer variation. Our best-performing models are segmental (semi-Markov) conditional random fields using deep neural network-based features. In the signer-dependent setting, our recognizers achieve up to about 92% letter accuracy. The multi-signer setting is much more challenging, but with neural network adaptation we achieve up to 83% letter accuracies in this setting.