# Continuous Sign Language Recognition Based on Motor attention mechanism and frame-level Self-distillation

arXiv: https://arxiv.org/abs/2402.19118 · HTML: https://mlsanity.com/paper/2402.19118/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 29 of its 56 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Improving Sign Language Translation with Monolingual Data by Sign Back-Translation](https://mlsanity.com/paper/2105.12397/) (2021)
- [Learning Lightweight Lane Detection CNNs by Self Attention Distillation](https://mlsanity.com/paper/1908.00821/) (2019)
- [STM: SpatioTemporal and Motion Encoding for Action Recognition](https://mlsanity.com/paper/1908.02486/) (2019)
- [Two-Stream Network for Sign Language Recognition and Translation](https://mlsanity.com/paper/2211.01367/) (2022)
- [Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2303.00440/) (2023)
- [Video-based Sign Language Recognition without Temporal Segmentation](https://mlsanity.com/paper/1801.10111/) (2018)
- [Visual Alignment Constraint for Continuous Sign Language Recognition](https://mlsanity.com/paper/2104.02330/) (2021)
- [Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2002.03187/) (2020)
- [Video Modeling with Correlation Networks](https://mlsanity.com/paper/1906.03349/) (2019)
- [SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding](https://mlsanity.com/paper/2305.04868/) (2023)
- [Fully Convolutional Networks for Continuous Sign Language Recognition](https://mlsanity.com/paper/2007.12402/) (2020)
- [Continuous Sign Language Recognition with Correlation Network](https://mlsanity.com/paper/2303.03202/) (2023)
- [Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation](https://mlsanity.com/paper/2103.08273/) (2021)
- [Spatio-Temporal Channel Correlation Networks for Action Classification](https://mlsanity.com/paper/1806.07754/) (2018)
- [Motion Feature Network: Fixed Motion Filter for Action Recognition](https://mlsanity.com/paper/1807.10037/) (2018)
- [Self-Emphasizing Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2211.17081/) (2022)
- [Sign Language Translation with Hierarchical Spatio-TemporalGraph Neural Network](https://mlsanity.com/paper/2111.07258/) (2021)
- [Alignment-guided Temporal Attention for Video Action Recognition](https://mlsanity.com/paper/2210.00132/) (2022)
- [Multi-Scale Local-Temporal Similarity Fusion for Continuous Sign Language Recognition](https://mlsanity.com/paper/2107.12762/) (2021)
- [Continuous sign language recognition based on cross-resolution knowledge distillation](https://mlsanity.com/paper/2303.06820/) (2023)

## Built on by (most cited first, 3 of 3)

- [A Chinese Continuous Sign Language Dataset Based on Complex Environments](https://mlsanity.com/paper/2409.11960/) (2024)
- [USTM: Unified Spatial and Temporal Modeling for Continuous Sign Language Recognition](https://mlsanity.com/paper/2512.13415/) (2025)
- [Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards](https://mlsanity.com/paper/2606.19352/) (2026)

## Abstract (arXiv)

Changes in facial expression, head movement, body movement and gesture movement are remarkable cues in sign language recognition, and most of the current continuous sign language recognition(CSLR) research methods mainly focus on static images in video sequences at the frame-level feature extraction stage, while ignoring the dynamic changes in the images. In this paper, we propose a novel motor attention mechanism to capture the distorted changes in local motion regions during sign language expression, and obtain a dynamic representation of image changes. And for the first time, we apply the self-distillation method to frame-level feature extraction for continuous sign language, which improves the feature expression without increasing the computational resources by self-distilling the features of adjacent stages and using the higher-order features as teachers to guide the lower-order features. The combination of the two constitutes our proposed holistic model of CSLR Based on motor attention mechanism and frame-level Self-Distillation (MAM-FSD), which improves the inference ability and robustness of the model. We conduct experiments on three publicly available datasets, and the experimental results show that our proposed method can effectively extract the sign language motion information in videos, improve the accuracy of CSLR and reach the state-of-the-art level.