# Fine-Tuning Video Transformers for Word-Level Bangla Sign Language: A Comparative Analysis for Classification Tasks

arXiv: https://arxiv.org/abs/2506.04367 · HTML: https://mlsanity.com/paper/2506.04367/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 78 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [Long-Term Feature Banks for Detailed Video Understanding](https://mlsanity.com/paper/1812.05038/) (2018)
- [Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison](https://mlsanity.com/paper/1910.11006/) (2019)
- [A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation](https://mlsanity.com/paper/2203.04287/) (2022)
- [MGMAE: Motion Guided Masking for Video Masked Autoencoding](https://mlsanity.com/paper/2308.10794/) (2023)
- [BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization](https://mlsanity.com/paper/2302.05075/) (2023)
- [Isolated Sign Language Recognition based on Tree Structure Skeleton Images](https://mlsanity.com/paper/2304.05403/) (2023)
- [LSA64: An Argentinian Sign Language Dataset](https://mlsanity.com/paper/2310.17429/) (2023)
- [Using Motion History Images with 3D Convolutional Networks in Isolated Sign Language Recognition](https://mlsanity.com/paper/2110.12396/) (2021)
- [MC-ViViT: Multi-branch Classifier-ViViT to detect Mild Cognitive Impairment in older adults using facial videos](https://mlsanity.com/paper/2304.05292/) (2023)
- [DeepASL: Enabling Ubiquitous and Non-Intrusive Word and Sentence-Level Sign Language Translation](https://mlsanity.com/paper/1802.07584/) (2018)
- [Self-Supervised Video Transformers for Isolated Sign Language Recognition](https://mlsanity.com/paper/2309.02450/) (2023)
- [BdSLW60: A Word-Level Bangla Sign Language Dataset](https://mlsanity.com/paper/2402.08635/) (2024)
- [Hierarchical Windowed Graph Attention Network and a Large Scale Dataset for Isolated Indian Sign Language Recognition](https://mlsanity.com/paper/2407.14224/) (2024)
- [A Deep Learning-based Multimodal Depth-Aware Dynamic Hand Gesture Recognition System](https://mlsanity.com/paper/2107.02543/) (2021)
- [A Comprehensive Review of Sign Language Recognition: Different Types, Modalities, and Datasets](https://mlsanity.com/paper/2204.03328/) (2022)
- [BdSLW401: Transformer-Based Word-Level Bangla Sign Language Recognition Using Relative Quantization Encoding (RQE)](https://mlsanity.com/paper/2503.02360/) (2025)
- [Transfer Learning for British Sign Language Modelling](https://mlsanity.com/paper/2006.02144/) (2020)
- [Fine-tuning of sign language recognition models: a technical report](https://mlsanity.com/paper/2302.07693/) (2023)

## Built on by (most cited first, 1 of 1)

- [Video-Based Sexual Harassment Detection Using Fine-Tuned Video Transformers](https://mlsanity.com/paper/2610.15354/) (2026)

## Abstract (arXiv)

Sign Language Recognition (SLR) involves the automatic identification and classification of sign gestures from images or video, converting them into text or speech to improve accessibility for the hearing-impaired community. In Bangladesh, Bangla Sign Language (BdSL) serves as the primary mode of communication for many individuals with hearing impairments. This study fine-tunes state-of-the-art video transformer architectures -- VideoMAE, ViViT, and TimeSformer -- on BdSLW60 (arXiv:2402.08635), a small-scale BdSL dataset with 60 frequent signs. We standardized the videos to 30 FPS, resulting in 9,307 user trial clips. To evaluate scalability and robustness, the models were also fine-tuned on BdSLW401 (arXiv:2503.02360), a large-scale dataset with 401 sign classes. Additionally, we benchmark performance against public datasets, including LSA64 and WLASL. Data augmentation techniques such as random cropping, horizontal flipping, and short-side scaling were applied to improve model robustness. To ensure balanced evaluation across folds during model selection, we employed 10-fold stratified cross-validation on the training set, while signer-independent evaluation was carried out using held-out test data from unseen users U4 and U8. Results show that video transformer models significantly outperform traditional machine learning and deep learning approaches. Performance is influenced by factors such as dataset size, video quality, frame distribution, frame rate, and model architecture. Among the models, the VideoMAE variant (MCG-NJU/videomae-base-finetuned-kinetics) achieved the highest accuracies of 95.5% on the frame rate corrected BdSLW60 dataset and 81.04% on the front-facing signs of BdSLW401 -- demonstrating strong potential for scalable and accurate BdSL recognition.