mlsanityData to 14 Oct 2026

FSboard: Over 3 million characters of ASL fingerspelling collected via smartphones

arXiv 2407.15806 ↗First posted 22 Jul 2024cs.CV cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Progress in machine understanding of sign languages has been slow and hampered by limited data. In this paper, we present FSboard, an American Sign Language fingerspelling dataset situated in a mobile text entry use case, collected from 147 paid and consenting Deaf signers using Pixel 4A selfie cameras in a variety of environments. Fingerspelling recognition is an incomplete solution that is only one small part of sign language translation, but it could provide some immediate benefit to Deaf/Hard of Hearing signers as more broadly capable technology develops. At >3 million characters in length and >250 hours in duration, FSboard is the largest fingerspelling recognition dataset to date by a factor of >10x. As a simple baseline, we finetune 30 Hz MediaPipe Holistic landmark inputs into ByT5-Small and achieve 11.1% Character Error Rate (CER) on a test set with unique phrases and signers. This quality degrades gracefully when decreasing frame rate and excluding face/body landmarks: plausible optimizations to help models run on device in real time.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 15 of its 55 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

15 of 55 references.

References · 15

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  3. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  4. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
  5. 2019Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspectivecited by 87
  6. 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 75
  7. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  8. 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 44
  9. 2023Sign Language Translation from Instructional Videoscited by 39
  10. 2024YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpuscited by 36
  11. 2019Fingerspelling recognition in the wild with iterative visual attentioncited by 28
  12. 2021Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognitioncited by 25
3 more
  1. 2018American Sign Language fingerspelling recognition in the wildcited by 20
  2. 2016Lexicon-Free Fingerspelling Recognition from Video: Data, Models, and Signer Adaptationcited by 6
  3. 2024Reconsidering Sentence-Level Sign Language Translationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025SignBind-LLM: Multi-Stage Modality Fusion for Sign Language Translationcited by 1
  2. 2026Bootstrapping Sign Language Annotations with Sign Language Modelscited by 1
  3. 2026OpenFS: Multi-Hand-Capable Fingerspelling Recognition with Implicit Signing-Hand Detection and Frame-Wise Letter-Conditioned Synthesiscited by 0
  4. 2026Recognising BSL Fingerspelling in Continuous Signing Sequencescited by 0
  5. 2026AdaDINO: Context-Adaptive DINO-Distilled Vision Foundation Models for Efficient Open-Vocabulary Edge Inferencecited by 0
  6. 2026Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standardscited by 0