# Self-Emphasizing Network for Continuous Sign Language Recognition

arXiv: https://arxiv.org/abs/2211.17081 · HTML: https://mlsanity.com/paper/2211.17081/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 30 Nov 2022.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond](https://mlsanity.com/paper/1904.11492/) (2019)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [3D Hand Shape and Pose from Images in the Wild](https://mlsanity.com/paper/1902.03451/) (2019)
- [Improving Sign Language Translation with Monolingual Data by Sign Back-Translation](https://mlsanity.com/paper/2105.12397/) (2021)
- [Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks](https://mlsanity.com/paper/1810.12348/) (2018)
- [Video-based Sign Language Recognition without Temporal Segmentation](https://mlsanity.com/paper/1801.10111/) (2018)
- [Visual Alignment Constraint for Continuous Sign Language Recognition](https://mlsanity.com/paper/2104.02330/) (2021)
- [Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2002.03187/) (2020)
- [SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition](https://mlsanity.com/paper/2110.05382/) (2021)
- [Fully Convolutional Networks for Continuous Sign Language Recognition](https://mlsanity.com/paper/2007.12402/) (2020)
- [Boosting Continuous Sign Language Recognition via Cross Modality Augmentation](https://mlsanity.com/paper/2010.05264/) (2020)
- [Pose-based Sign Language Recognition using GCN and BERT](https://mlsanity.com/paper/2012.00781/) (2020)
- [Mask-Guided Attention Network for Occluded Pedestrian Detection](https://mlsanity.com/paper/1910.06160/) (2019)
- [SF-Net: Structured Feature Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/1908.01341/) (2019)

## Built on by (most cited first, 30 of 33)

- [Continuous Sign Language Recognition with Correlation Network](https://mlsanity.com/paper/2303.03202/) (2023)
- [Uni-Sign: Toward Unified Sign Language Understanding at Scale](https://mlsanity.com/paper/2501.15187/) (2025)
- [A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars](https://mlsanity.com/paper/2401.04730/) (2024)
- [AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognition](https://mlsanity.com/paper/2308.08327/) (2023)
- [SlowFast Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2309.12304/) (2023)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)
- [CorrNet+: Sign Language Recognition and Translation via Spatial-Temporal Correlation](https://mlsanity.com/paper/2404.11111/) (2024)
- [Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition](https://mlsanity.com/paper/2406.10501/) (2024)
- [EvSign: Sign Language Recognition and Translation with Streaming Events](https://mlsanity.com/paper/2407.12593/) (2024)
- [Isharah: A Large-Scale Multi-Scene Dataset for Continuous Sign Language Recognition](https://mlsanity.com/paper/2506.03615/) (2025)
- [SignVTCL: Multi-Modal Continuous Sign Language Recognition Enhanced by Visual-Textual Contrastive Learning](https://mlsanity.com/paper/2401.11847/) (2024)
- [CLIP-SLA: Parameter-Efficient CLIP Adaptation for Continuous Sign Language Recognition](https://mlsanity.com/paper/2504.01666/) (2025)
- [A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation](https://mlsanity.com/paper/2308.08849/) (2023)
- [Continuous Sign Language Recognition Based on Motor attention mechanism and frame-level Self-distillation](https://mlsanity.com/paper/2402.19118/) (2024)
- [Improving Continuous Sign Language Recognition with Adapted Image Models](https://mlsanity.com/paper/2404.08226/) (2024)
- [A Chinese Continuous Sign Language Dataset Based on Complex Environments](https://mlsanity.com/paper/2409.11960/) (2024)
- [Continuous Sign Language Recognition with Adapted Conformer via Unsupervised Pretraining](https://mlsanity.com/paper/2405.12018/) (2024)
- [A Comparative Study of Continuous Sign Language Recognition Techniques](https://mlsanity.com/paper/2406.12369/) (2024)
- [A Signer-Invariant Conformer and Multi-Scale Fusion Transformer for Continuous Sign Language Recognition](https://mlsanity.com/paper/2508.09372/) (2025)
- [Large Sign Language Models: Toward 3D American Sign Language Translation](https://mlsanity.com/paper/2511.08535/) (2025)
- [USTM: Unified Spatial and Temporal Modeling for Continuous Sign Language Recognition](https://mlsanity.com/paper/2512.13415/) (2025)
- [Denoising-Contrastive Alignment for Continuous Sign Language Recognition](https://mlsanity.com/paper/2305.03614/) (2023)
- [SignAttention: On the Interpretability of Transformer Models for Sign Language Translation](https://mlsanity.com/paper/2410.14506/) (2024)
- [Hierarchical Sub-action Tree for Continuous Sign Language Recognition](https://mlsanity.com/paper/2506.20947/) (2025)
- [DESign: Dynamic Context-Aware Convolution and Efficient Subnet Regularization for Continuous Sign Language Recognition](https://mlsanity.com/paper/2507.03339/) (2025)
- [AutoSign: Direct Pose-to-Text Translation for Continuous Sign Language Recognition](https://mlsanity.com/paper/2507.19840/) (2025)
- [SSL-SSAW: Self-Supervised Learning with Sigmoid Self-Attention Weighting for Question-Based Sign Language Translation](https://mlsanity.com/paper/2509.14036/) (2025)
- [Towards Multi-View Sign Language Understanding: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2604.18184/) (2026)
- [Do Transformer Temporal Heads and Post-Pooling Motion Gates Help CorrNet-based CSLR? An Empirical Study](https://mlsanity.com/paper/2607.09890/) (2026)
- [ODE-Based Transformer Decoders for Iterative Sign Language Translation](https://mlsanity.com/paper/2608.11352/) (2026)

## Abstract (arXiv)

Hand and face play an important role in expressing sign language. Their features are usually especially leveraged to improve system performance. However, to effectively extract visual representations and capture trajectories for hands and face, previous methods always come at high computations with increased training complexity. They usually employ extra heavy pose-estimation networks to locate human body keypoints or rely on additional pre-extracted heatmaps for supervision. To relieve this problem, we propose a self-emphasizing network (SEN) to emphasize informative spatial regions in a self-motivated way, with few extra computations and without additional expensive supervision. Specifically, SEN first employs a lightweight subnetwork to incorporate local spatial-temporal features to identify informative regions, and then dynamically augment original features via attention maps. It's also observed that not all frames contribute equally to recognition. We present a temporal self-emphasizing module to adaptively emphasize those discriminative frames and suppress redundant ones. A comprehensive comparison with previous methods equipped with hand and face features demonstrates the superiority of our method, even though they always require huge computations and rely on expensive extra supervision. Remarkably, with few extra computations, SEN achieves new state-of-the-art accuracy on four large-scale datasets, PHOENIX14, PHOENIX14-T, CSL-Daily, and CSL. Visualizations verify the effects of SEN on emphasizing informative spatial and temporal features. Code is available at https://github.com/hulianyuyy/SEN_CSLR