# AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methods

arXiv: https://arxiv.org/abs/2008.00932 · HTML: https://mlsanity.com/paper/2008.00932/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2020.

## Built on by (most cited first, 30 of 75)

- [Skeleton Aware Multi-modal Sign Language Recognition](https://mlsanity.com/paper/2103.08833/) (2021)
- [A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation](https://mlsanity.com/paper/2203.04287/) (2022)
- [Including Signed Languages in Natural Language Processing](https://mlsanity.com/paper/2105.05222/) (2021)
- [Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2405.04164/) (2024)
- [Sign Language Translation from Instructional Videos](https://mlsanity.com/paper/2304.06371/) (2023)
- [BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization](https://mlsanity.com/paper/2302.05075/) (2023)
- [OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages](https://mlsanity.com/paper/2110.05877/) (2021)
- [SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark](https://mlsanity.com/paper/2310.20436/) (2023)
- [BBC-Oxford British Sign Language Dataset](https://mlsanity.com/paper/2111.03635/) (2021)
- [Lost in Translation, Found in Context: Sign Language Translation with Contextual Cues](https://mlsanity.com/paper/2501.09754/) (2025)
- [Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognition](https://mlsanity.com/paper/2104.10166/) (2021)
- [Ham2Pose: Animating Sign Language Notation into Pose Sequences](https://mlsanity.com/paper/2211.13613/) (2022)
- [Machine Translation from Signed to Spoken Languages: State of the Art and Challenges](https://mlsanity.com/paper/2202.03086/) (2022)
- [Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble](https://mlsanity.com/paper/2110.06161/) (2021)
- [Isolated Sign Language Recognition based on Tree Structure Skeleton Images](https://mlsanity.com/paper/2304.05403/) (2023)
- [Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse](https://mlsanity.com/paper/2308.09622/) (2023)
- [SignRep: Enhancing Self-Supervised Sign Representations](https://mlsanity.com/paper/2503.08529/) (2025)
- [MASA: Motion-aware Masked Autoencoder with Semantic Alignment for Sign Language Recognition](https://mlsanity.com/paper/2405.20666/) (2024)
- [Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition](https://mlsanity.com/paper/2110.12396/) (2021)
- [ChaLearn LAP Large Scale Signer Independent Isolated Sign Language Recognition Challenge: Design, Results and Future Research](https://mlsanity.com/paper/2105.05066/) (2021)
- [Survey on Hand Gesture Recognition from Visual Input](https://mlsanity.com/paper/2501.11992/) (2025)
- [Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Field](https://mlsanity.com/paper/2603.13240/) (2026)
- [Slovo: Russian Sign Language Dataset](https://mlsanity.com/paper/2305.14527/) (2023)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)
- [Resources for Turkish Natural Language Processing: A critical survey](https://mlsanity.com/paper/2204.05042/) (2022)
- [BdSLW60: A Word-Level Bangla Sign Language Dataset](https://mlsanity.com/paper/2402.08635/) (2024)
- [A Hong Kong Sign Language Corpus Collected from Sign-interpreted TV News](https://mlsanity.com/paper/2405.00980/) (2024)
- [Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition](https://mlsanity.com/paper/2406.10501/) (2024)
- [iSign: A Benchmark for Indian Sign Language Processing](https://mlsanity.com/paper/2407.05404/) (2024)
- [Hierarchical Windowed Graph Attention Network and a Large Scale Dataset for Isolated Indian Sign Language Recognition](https://mlsanity.com/paper/2407.14224/) (2024)

## Abstract (arXiv)

Sign language recognition is a challenging problem where signs are identified by simultaneous local and global articulations of multiple sources, i.e. hand shape and orientation, hand movements, body posture, and facial expressions. Solving this problem computationally for a large vocabulary of signs in real life settings is still a challenge, even with the state-of-the-art models. In this study, we present a new largescale multi-modal Turkish Sign Language dataset (AUTSL) with a benchmark and provide baseline models for performance evaluations. Our dataset consists of 226 signs performed by 43 different signers and 38,336 isolated sign video samples in total. Samples contain a wide variety of backgrounds recorded in indoor and outdoor environments. Moreover, spatial positions and the postures of signers also vary in the recordings. Each sample is recorded with Microsoft Kinect v2 and contains RGB, depth, and skeleton modalities. We prepared benchmark training and test sets for user independent assessments of the models. We trained several deep learning based models and provide empirical evaluations using the benchmark; we used CNNs to extract features, unidirectional and bidirectional LSTM models to characterize temporal information. We also incorporated feature pooling modules and temporal attention to our models to improve the performances. We evaluated our baseline models on AUTSL and Montalbano datasets. Our models achieved competitive results with the state-of-the-art methods on Montalbano dataset, i.e. 96.11% accuracy. In AUTSL random train-test splits, our models performed up to 95.95% accuracy. In the proposed user-independent benchmark dataset our best baseline model achieved 62.02% accuracy. The gaps in the performances of the same baseline models show the challenges inherent in our benchmark dataset. AUTSL benchmark dataset is publicly available at https://cvml.ankara.edu.tr.