# Hierarchical Windowed Graph Attention Network and a Large Scale Dataset for Isolated Indian Sign Language Recognition

arXiv: https://arxiv.org/abs/2407.14224 · HTML: https://mlsanity.com/paper/2407.14224/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 56 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison](https://mlsanity.com/paper/1910.11006/) (2019)
- [MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language](https://mlsanity.com/paper/1812.01053/) (2018)
- [Skeleton Aware Multi-modal Sign Language Recognition](https://mlsanity.com/paper/2103.08833/) (2021)
- [Video-based Sign Language Recognition without Temporal Segmentation](https://mlsanity.com/paper/1801.10111/) (2018)
- [AUTSL: A Large Scale Multi-modal Turkish Sign Language Dataset and Baseline Methods](https://mlsanity.com/paper/2008.00932/) (2020)
- [YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss](https://mlsanity.com/paper/2204.06806/) (2022)
- [Natural Language-Assisted Sign Language Recognition](https://mlsanity.com/paper/2303.12080/) (2023)
- [Watch, read and lookup: learning to spot signs from multiple supervisors](https://mlsanity.com/paper/2010.04002/) (2020)
- [OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages](https://mlsanity.com/paper/2110.05877/) (2021)
- [DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks](https://mlsanity.com/paper/1907.11065/) (2019)
- [BosphorusSign22k Sign Language Recognition Dataset](https://mlsanity.com/paper/2004.01283/) (2020)
- [Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble](https://mlsanity.com/paper/2110.06161/) (2021)

## Built on by (most cited first, 6 of 6)

- [Logos as a Well-Tempered Pre-train for Sign Language Recognition](https://mlsanity.com/paper/2505.10481/) (2025)
- [Fine-Tuning Video Transformers for Word-Level Bangla Sign Language: A Comparative Analysis for Classification Tasks](https://mlsanity.com/paper/2506.04367/) (2025)
- [SignBart -- New approach with the skeleton sequence for Isolated Sign language Recognition](https://mlsanity.com/paper/2506.21592/) (2025)
- [Stack Transformer Based Spatial-Temporal Attention Model for Dynamic Sign Language and Fingerspelling Recognition](https://mlsanity.com/paper/2503.16855/) (2025)
- [HandCraft: Dynamic Sign Generation for Synthetic Data Augmentation](https://mlsanity.com/paper/2508.14345/) (2025)
- [Skeleton-based sign language recognition using a dual-stream spatio-temporal dynamic graph convolutional network](https://mlsanity.com/paper/2509.08661/) (2025)

## Abstract (arXiv)

Automatic Sign Language (SL) recognition is an important task in the computer vision community. To build a robust SL recognition system, we need a considerable amount of data which is lacking particularly in Indian sign language (ISL). In this paper, we introduce a large-scale isolated ISL dataset and a novel SL recognition model based on skeleton graph structure. The dataset covers 2002 daily used common words in the deaf community recorded by 20 (10 male and 10 female) deaf adult signers (contains 40033 videos). We propose a SL recognition model namely Hierarchical Windowed Graph Attention Network (HWGAT) by utilizing the human upper body skeleton graph. The HWGAT tries to capture distinctive motions by giving attention to different body parts induced by the human skeleton graph. The utility of the proposed dataset and the usefulness of our model are evaluated through extensive experiments. We pre-trained the proposed model on the presented dataset and fine-tuned it across different sign language datasets further boosting the performance of 1.10, 0.46, 0.78, and 6.84 percentage points on INCLUDE, LSA64, AUTSL and WLASL respectively compared to the existing state-of-the-art keypoints-based models.