SIGNET: Motion-Level Knowledge Transfer for Cross-Language Sign Language Translation
Not on the map. Why?
Abstract (arXiv)
Sign language translation (SLT) remains challenging due to its high spatio-temporal complexity, long sequences, and the need to model multiple articulators without relying on gloss annotations. Existing approaches are typically tailored to individual datasets or languages and struggle to scale, while overlooking the relationships between sign languages that could inform more effective cross-lingual transfer. We present SIGNET, a framework that enables motion-level knowledge transfer for cross-language sign language translation. Our key insight is that, although sign languages differ in grammar and lexicon, pretrained models capture motion-level visual patterns that can be reused across datasets and languages. SIGNET integrates multiple pretrained sign language backbones through an attention-based, hand-prior aggregation mechanism that guides a gated fusion network in dynamically selecting the most relevant experts. Comprehensive experiments on four benchmarks (How2Sign, Phoenix14T, CSL-Daily, and MeineDGS) demonstrate state-of-the-art translation performance, and SIGNET also surpasses prior methods on WLASL for sign language recognition.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 51 of its 62 references that are in the map. First posted on arXiv 26 Jun 2026.
Builds on
51 of 62 references.
References · 51
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
- 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
- 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
38 more
- 2022Mixture-of-Experts with Expert Choice Routingcited by 611
- 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
- 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 217
- 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
- 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 172
- 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 170
- 2022Two-Stream Network for Sign Language Recognition and Translationcited by 122
- 2023Gloss-free Sign Language Translation: Improving from Visual-Language Pretrainingcited by 104
- 2022A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translationcited by 80
- 2024LLMs are Good Sign Language Translatorscited by 76
- 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 74
- 2024Sign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translationcited by 56
- 2025Uni-Sign: Toward Unified Sign Language Understanding at Scalecited by 55
- 2021Spatio-temporal Relation Modeling for Few-shot Action Recognitioncited by 52
- 2023Natural Language-Assisted Sign Language Recognitioncited by 46
- 2024Towards Privacy-Aware Sign Language Translation at Scalecited by 42
- 2023Sign Language Translation from Instructional Videoscited by 39
- 2024YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpuscited by 36
- 2023SLTUNET: A Simple Unified Model for Sign Language Translationcited by 35
- 2024Improving Gloss-free Sign Language Translation by Reducing Representation Densitycited by 30
- 2021BBC-Oxford British Sign Language Datasetcited by 27
- 2021SimulSLT: End-to-End Simultaneous Sign Language Translationcited by 26
- 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 25
- 2024Factorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translationcited by 23
- 2024C2RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrievalcited by 23
- 2023Sign Language Translation with Iterative Prototypecited by 20
- 2024SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scalecited by 18
- 2024MoE-LPR: Multilingual Extension of Large Language Models through Mixture-of-Experts with Language Priors Routingcited by 16
- 2025SignRep: Enhancing Self-Supervised Sign Representationscited by 15
- 2024LLaVA-SLT: Visual Language Tuning for Sign Language Translationcited by 14
- 2024Leveraging the Power of MLLMs for Gloss-Free Sign Language Translationcited by 13
- 2025Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translationcited by 12
- 2025Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translationcited by 11
- 2025VALLR: Visual ASR Language Model for Lip Readingcited by 10
- 2026Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Fieldcited by 9
- 2025Beyond Gloss: A Hand-Centric Framework for Gloss-Free Sign Language Translationcited by 7
- 2023CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrievalcited by 6
- 2025Hierarchical Feature Alignment for Gloss-Free Sign Language Translationcited by 6
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.