mlsanityData to 14 Oct 2026

Skeletal Graph Self-Attention: Embedding a Skeleton Inductive Bias into Sign Language Production

arXiv 2112.05277 ↗First posted 6 Dec 2021cs.CV cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Recent approaches to Sign Language Production (SLP) have adopted spoken language Neural Machine Translation (NMT) architectures, applied without sign-specific modifications. In addition, these works represent sign language as a sequence of skeleton pose vectors, projected to an abstract representation with no inherent skeletal structure. In this paper, we represent sign language sequences as a skeletal graph structure, with joints as nodes and both spatial and temporal connections as edges. To operate on this graphical structure, we propose Skeletal Graph Self-Attention (SGSA), a novel graphical attention layer that embeds a skeleton inductive bias into the SLP model. Retaining the skeletal feature representation throughout, we directly apply a spatio-temporal adjacency matrix into the self-attention formulation. This provides structure and context to each skeletal joint that is not possible when using a non-graphical abstract representation, enabling fluid and expressive sign language production. We evaluate our Skeletal Graph Self-Attention architecture on the challenging RWTH-PHOENIX-Weather-2014T(PHOENIX14T) dataset, achieving state-of-the-art back translation performance with an 8% and 7% improvement over competing methods for the dev and test sets.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 41 of its 77 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

41 of 77 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2017Graph Attention Networkscited by 8,923
  5. 2016Layer Normalizationcited by 5,669
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  9. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  10. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  11. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  12. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
29 more
  1. 2020A Generalization of Transformer Networks to Graphscited by 661
  2. 2019Graph Transformer Networkscited by 523
  3. 2021Rethinking Graph Transformers with Spectral Attentioncited by 473
  4. 2018Graph Convolutional Networks for Text Classificationcited by 345
  5. 2018Generating Wikipedia by Summarizing Long Sequencescited by 335
  6. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 256
  7. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
  8. 2018Learning Deep Generative Models of Graphscited by 229
  9. 2019Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networkscited by 229
  10. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  11. 2018Learning Human-Object Interactions by Graph Parsing Neural Networkscited by 176
  12. 2019Text Generation from Knowledge Graphs with Graph Transformerscited by 122
  13. 2020Progressive Transformers for End-to-End Sign Language Productioncited by 120
  14. 2021Skeleton Aware Multi-modal Sign Language Recognitioncited by 101
  15. 2021TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Trackingcited by 93
  16. 2019Relational Graph Attention Networkscited by 82
  17. 2019Session-based Social Recommendation via Dynamic Graph Attention Networkscited by 76
  18. 2020Multi-channel Transformers for Multi-articulatory Sign Language Translationcited by 71
  19. 2019Explicit Sparse Transformer: Concentrated Attention Through Explicit Selectioncited by 62
  20. 2020Adversarial Training for Multi-Channel Sign Language Productioncited by 57
  21. 2021Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networkscited by 50
  22. 2021Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitivescited by 50
  23. 2018Modeling Localness for Self-Attention Networkscited by 43
  24. 2019Joey NMT: A Minimalist NMT Toolkit for Novicescited by 39
  25. 2020Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Videocited by 34
  26. 2021Content4All Open Research Sign Language Translation Datasetscited by 30
  27. 2019Spatial-Temporal Graph Convolutional Networks for Sign Language Recognitioncited by 23
  28. 2021AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video Anonymisationcited by 16
  29. 2020Better Sign Language Translation with STMC-Transformercited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 4; 2025: 3.

11 papers.

  1. 2022Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Productioncited by 61
  2. 2023Sign Language Translation from Instructional Videoscited by 39
  3. 2023SignDiff: Diffusion Model for American Sign Language Productioncited by 25
  4. 2024Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Productioncited by 16
  5. 2024Pose-Guided Fine-Grained Sign Language Video Generationcited by 8
  6. 2025SignX: Continuous Sign Recognition in Compact Pose-Rich Latent Spacecited by 4
  7. 2024SignLLM: Sign Language Production Large Language Modelscited by 2
  8. 2024The NGT200 Dataset: Geometric Multi-View Isolated Sign Recognitioncited by 2
  9. 2022Topic Detection in Continuous Sign Language Videoscited by 1
  10. 2025Beyond Words: AuralLLM and SignMST-C for Sign Language Production and Bidirectional Accessibilitycited by 0
  11. 2025Towards Skeletal and Signer Noise Reduction in Sign Language Production via Quaternion-Based Pose Encoding and Contrastive Learningcited by 0