mlsanityData to 7 Oct 2026

Multi-channel Transformers for Multi-articulatory Sign Language Translation

arXiv 2009.00299 ↗First posted 1 Sep 2020cs.CVcited by 71

Not on the map. Why?

Abstract (arXiv)

Sign languages use multiple asynchronous information channels (articulators), not just the hands but also the face and body, which computational approaches often ignore. In this paper we tackle the multi-articulatory sign language translation task and propose a novel multi-channel transformer architecture. The proposed architecture allows both the inter and intra contextual relationships between different sign articulators to be modelled within the transformer network itself, while also maintaining channel specific information. We evaluate our approach on the RWTH-PHOENIX-Weather-2014T dataset and report competitive translation performance. Importantly, we overcome the reliance on gloss annotations which underpin other state-of-the-art approaches, thereby removing future need for expensive curated datasets.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 15; 2023: 17; 2024: 12; 2025: 8; 2026: 4.

71 papers.

Sort:

By area:

  1. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
  2. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 168
  3. 2022Two-Stream Network for Sign Language Recognition and Translationcited by 122
  4. 2022A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translationcited by 80
  5. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  6. 2021Including Signed Languages in Natural Language Processingcited by 67
  7. 2022Video Transformers: A Surveycited by 63
  8. 2023SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understandingcited by 59
  9. 2021Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networkscited by 50
  10. 2021Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitivescited by 50
  11. 2023Gloss Attention for Gloss-free Sign Language Translationcited by 50
  12. 2023Sign Language Translation from Instructional Videoscited by 39
38 more
  1. 2022G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Modelcited by 38
  2. 2021Stochastic Transformer Networks with Linear Competing Units: Application to end-to-end SL Translationcited by 35
  3. 2023SLTUNET: A Simple Unified Model for Sign Language Translationcited by 35
  4. 2021Read and Attend: Temporal Localisation in Sign Language Videoscited by 34
  5. 2023CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learningcited by 32
  6. 2021Content4All Open Research Sign Language Translation Datasetscited by 30
  7. 2024Improving Gloss-free Sign Language Translation by Reducing Representation Densitycited by 30
  8. 2023Improving Continuous Sign Language Recognition with Cross-Lingual Signscited by 27
  9. 2021Sign Language Production: A Reviewcited by 26
  10. 2021Skeletor: Skeletal Transformers for Robust Body-Pose Estimationcited by 26
  11. 2021SimulSLT: End-to-End Simultaneous Sign Language Translationcited by 26
  12. 2021Evaluating the Immediate Applicability of Pose Estimation for Sign Language Recognitioncited by 25
  13. 2022Considerations for meaningful sign language machine translation based on glossescited by 24
  14. 2023Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignmentcited by 22
  15. 2022Machine Translation from Signed to Spoken Languages: State of the Art and Challengescited by 20
  16. 2022Automatic dense annotation of large-vocabulary sign language videoscited by 20
  17. 2023Sign Language Translation with Iterative Prototypecited by 20
  18. 2024A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatarscited by 18
  19. 2024SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scalecited by 18
  20. 2022Machine Translation between Spoken Languages and Signed Languages Represented in SignWritingcited by 17
  21. 2023Cross-modality Data Augmentation for End-to-End Sign Language Translationcited by 16
  22. 2024Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observationcited by 16
  23. 2023Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discoursecited by 15
  24. 2024A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervisioncited by 15
  25. 2023Sign Language Production with Latent Motion Transformercited by 13
  26. 2021PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolutioncited by 11
  27. 2021Skeletal Graph Self-Attention: Embedding a Skeleton Inductive Bias into Sign Language Productioncited by 11
  28. 2022A Token-level Contrastive Framework for Sign Language Translationcited by 10
  29. 2024Multi-Stream Keypoint Attention Network for Sign Language Recognition and Translationcited by 9
  30. 2022All You Need In Sign Language Productioncited by 8
  31. 2024From CNNs to Transformers in Multimodal Human Action Recognition: A Surveycited by 7
  32. 2023A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Datasetcited by 6
  33. 2025Deep Understanding of Sign Language for Sign to Subtitle Alignmentcited by 6
  34. 2023Learnt Contrastive Concept Embeddings for Sign Recognitioncited by 5
  35. 2025Text-Driven 3D Hand Motion Generation from Sign Language Datacited by 5
  36. 2022Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translationcited by 4
  37. 2022Leveraging Graph-based Cross-modal Information Fusion for Neural Sign Language Translationcited by 4
  38. 2024SEDS: Semantically Enhanced Dual-Stream Encoder for Sign Language Retrievalcited by 4

50 most cited of 71 All 71 (JSON)