# ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition

arXiv: https://arxiv.org/abs/2304.05934 · HTML: https://mlsanity.com/paper/2304.05934/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 66 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 13 of 13)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison](https://mlsanity.com/paper/1910.11006/) (2019)
- [MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language](https://mlsanity.com/paper/1812.01053/) (2018)
- [Skeleton Aware Multi-modal Sign Language Recognition](https://mlsanity.com/paper/2103.08833/) (2021)
- [Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective](https://mlsanity.com/paper/1908.08597/) (2019)
- [Including Signed Languages in Natural Language Processing](https://mlsanity.com/paper/2105.05222/) (2021)
- [Quantitative Survey of the State of the Art in Sign Language Recognition](https://mlsanity.com/paper/2008.09918/) (2020)
- [Natural Language-Assisted Sign Language Recognition](https://mlsanity.com/paper/2303.12080/) (2023)
- [OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages](https://mlsanity.com/paper/2110.05877/) (2021)

## Built on by (most cited first, 30 of 34)

- [The Sem-Lex Benchmark: Modeling ASL Signs and Their Phonemes](https://mlsanity.com/paper/2310.00196/) (2023)
- [SignRep: Enhancing Self-Supervised Sign Representations](https://mlsanity.com/paper/2503.08529/) (2025)
- [Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Field](https://mlsanity.com/paper/2603.13240/) (2026)
- [MM-WLAuslan: Multi-View Multi-Modal Word-Level Australian Sign Language Recognition Dataset](https://mlsanity.com/paper/2410.19488/) (2024)
- [DSTSA-GCN: Advancing Skeleton-Based Gesture Recognition with Semantic-Aware Spatio-Temporal Topology Modeling](https://mlsanity.com/paper/2501.12086/) (2025)
- ["It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models](https://mlsanity.com/paper/2511.08917/) (2025)
- [SignVLA: A Gloss-Free Vision-Language-Action Framework for Real-Time Sign Language-Guided Robotic Manipulation](https://mlsanity.com/paper/2602.22514/) (2026)
- [The NGT200 Dataset: Geometric Multi-View Isolated Sign Recognition](https://mlsanity.com/paper/2409.15284/) (2024)
- [Logos as a Well-Tempered Pre-train for Sign Language Recognition](https://mlsanity.com/paper/2505.10481/) (2025)
- [Exploring Pose-based Sign Language Translation: Ablation Studies and Attention Insights](https://mlsanity.com/paper/2507.01532/) (2025)
- [AzSLD: Azerbaijani Sign Language Dataset for Fingerspelling, Word, and Sentence Translation with Baseline Software](https://mlsanity.com/paper/2411.12865/) (2024)
- [SignBart -- New approach with the skeleton sequence for Isolated Sign language Recognition](https://mlsanity.com/paper/2506.21592/) (2025)
- [SignAgent: Agentic LLMs for Linguistically-Grounded Sign Language Annotation and Dataset Curation](https://mlsanity.com/paper/2603.19059/) (2026)
- [Bootstrapping Sign Language Annotations with Sign Language Models](https://mlsanity.com/paper/2604.07606/) (2026)
- [SignDATA: Data Pipeline for Sign Language Translation](https://mlsanity.com/paper/2604.20357/) (2026)
- [Representing Signs as Signs: One-Shot ISLR to Facilitate Functional Sign Language Technologies](https://mlsanity.com/paper/2502.20171/) (2025)
- [Towards an AI-Driven Video-Based American Sign Language Dictionary: Exploring Design and Usage Experience with Learners](https://mlsanity.com/paper/2504.05857/) (2025)
- [SSL-SLR: Self-Supervised Representation Learning for Sign Language Recognition](https://mlsanity.com/paper/2509.05188/) (2025)
- [IsoSignVid2Aud: Sign Language Video to Audio Conversion without Text Intermediaries](https://mlsanity.com/paper/2510.07837/) (2025)
- [Isolated Sign Language Recognition with Segmentation and Pose Estimation](https://mlsanity.com/paper/2512.14876/) (2025)
- [State Space Models are Effective Sign Language Learners: Exploiting Phonological Compositionality for Vocabulary-Scale Recognition](https://mlsanity.com/paper/2604.08761/) (2026)
- [Towards Multi-View Sign Language Understanding: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2604.18184/) (2026)
- [Cripping AI: Reimagining AI Through Lived Disability Experiences](https://mlsanity.com/paper/2605.02080/) (2026)
- [Sign Language Recognition and Translation for Low-Resource Languages: Challenges and Pathways Forward](https://mlsanity.com/paper/2605.12096/) (2026)
- [SignVLA: Real-Time Sign Language-Guided Robotic Manipulation via Attention LSTM and Vision-Language-Action Models](https://mlsanity.com/paper/2606.20857/) (2026)
- [Phonological Perception of Sign Language Models](https://mlsanity.com/paper/2606.28667/) (2026)
- [Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed](https://mlsanity.com/paper/2607.24898/) (2026)
- [JSL-DC: A Word-Level Japanese Sign Language Dataset with Linguist-Derived Descriptions for Distinguishing Confusable Signs](https://mlsanity.com/paper/2608.18412/) (2026)
- [SignMatch: Matching Dictionary Signs to Continuous Sign Language Video](https://mlsanity.com/paper/2609.01886/) (2026)
- [SignSeek: Learning Transferable Representations for Sign Dictionary Retrieval](https://mlsanity.com/paper/2609.03695/) (2026)

## Abstract (arXiv)

Sign languages are used as a primary language by approximately 70 million D/deaf people world-wide. However, most communication technologies operate in spoken and written languages, creating inequities in access. To help tackle this problem, we release ASL Citizen, the first crowdsourced Isolated Sign Language Recognition (ISLR) dataset, collected with consent and containing 83,399 videos for 2,731 distinct signs filmed by 52 signers in a variety of environments. We propose that this dataset be used for sign language dictionary retrieval for American Sign Language (ASL), where a user demonstrates a sign to their webcam to retrieve matching signs from a dictionary. We show that training supervised machine learning classifiers with our dataset advances the state-of-the-art on metrics relevant for dictionary retrieval, achieving 63% accuracy and a recall-at-10 of 91%, evaluated entirely on videos of users who are not present in the training or validation sets. An accessible PDF of this article is available at the following link: https://aashakadesai.github.io/research/ASLCitizen_arxiv_updated.pdf