mlsanityData to 14 Oct 2026

Graph Network for Sign Language Tasks

arXiv 2504.12020 ↗First posted 16 Apr 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in sign language research have benefited from CNN-based backbones, which are primarily transferred from traditional computer vision tasks (eg object identification, image recognition). However, these CNN-based backbones usually excel at extracting features like contours and texture, but may struggle with capturing sign-related features. In fact, sign language tasks require focusing on sign-related regions, including the collaboration between different regions (eg left hand region and right hand region) and the effective content in a single region. To capture such region-related features, we introduce MixSignGraph, which represents sign sequences as a group of mixed graphs and designs the following three graph modules for feature extraction, ie Local Sign Graph (LSG) module, Temporal Sign Graph (TSG) module and Hierarchical Sign Graph (HSG) module. Specifically, the LSG module learns the correlation of intra-frame cross-region features within one frame, ie focusing on spatial features. The TSG module tracks the interaction of inter-frame cross-region features among adjacent frames, ie focusing on temporal features. The HSG module aggregates the same-region features from different-granularity feature maps of a frame, ie focusing on hierarchical features. In addition, to further improve the performance of sign language tasks without gloss annotations, we propose a simple yet counter-intuitive Text-driven CTC Pre-training (TCP) method, which generates pseudo gloss labels from text labels for model pre-training. Extensive experiments conducted on current five public sign language datasets demonstrate the superior performance of the proposed model. Notably, our model surpasses the SOTA models on multiple sign language tasks across several datasets, without relying on any additional cues.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 67 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

42 of 67 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2017Inductive Representation Learning on Large Graphscited by 6,320
  9. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  10. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  11. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  12. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
30 more
  1. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  2. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  3. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  4. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  5. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  6. 2021How Attentive are Graph Attention Networks?cited by 835
  7. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 680
  8. 2021Early Convolutions Help Transformers See Bettercited by 396
  9. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  10. 2022Vision GNN: An Image is Worth Graph of Nodescited by 186
  11. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 179
  12. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 173
  13. 2022Two-Stream Network for Sign Language Recognition and Translationcited by 123
  14. 2023Gloss-free Sign Language Translation: Improving from Visual-Language Pretrainingcited by 105
  15. 2021Visual Alignment Constraint for Continuous Sign Language Recognitioncited by 80
  16. 2022A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translationcited by 80
  17. 2020TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translationcited by 75
  18. 2020Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognitioncited by 74
  19. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  20. 2020Multi-channel Transformers for Multi-articulatory Sign Language Translationcited by 71
  21. 2020Tackling Over-Smoothing for General Graph Convolutional Networkscited by 66
  22. 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
  23. 2020Fully Convolutional Networks for Continuous Sign Language Recognitioncited by 58
  24. 2024Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translationcited by 57
  25. 2023Continuous Sign Language Recognition with Correlation Networkcited by 56
  26. 2023Gloss Attention for Gloss-free Sign Language Translationcited by 51
  27. 2020Boosting Continuous Sign Language Recognition via Cross Modality Augmentationcited by 40
  28. 2023Sign Language Translation from Instructional Videoscited by 39
  29. 2023Improving Continuous Sign Language Recognition with Cross-Lingual Signscited by 27
  30. 2021Sign Language Translation with Hierarchical Spatio-TemporalGraph Neural Networkcited by 26

Built on by

Not cited yet.