mlsanityData to 7 Oct 2026

Gloss-Free End-to-End Sign Language Translation

arXiv 2305.12876 ↗First posted 22 May 2023cs.CV cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

In this paper, we tackle the problem of sign language translation (SLT) without gloss annotations. Although intermediate representation like gloss has been proven effective, gloss annotations are hard to acquire, especially in large quantities. This limits the domain coverage of translation datasets, thus handicapping real-world applications. To mitigate this problem, we design the Gloss-Free End-to-end sign language translation framework (GloFE). Our method improves the performance of SLT in the gloss-free setting by exploiting the shared underlying semantics of signs and the corresponding spoken translation. Common concepts are extracted from the text and used as a weak form of intermediate representation. The global embedding of these concepts is used as a query for cross-attention to find the corresponding information within the learned visual features. In a contrastive manner, we encourage the similarity of query results between samples containing such concepts and decrease those that do not. We obtained state-of-the-art results on large-scale datasets, including OpenASL and How2Sign. The code and model will be available at https://github.com/HenryLittle/GloFE.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

30 of 49 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2016Layer Normalizationcited by 5,579
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  10. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  11. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  12. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
18 more
  1. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  2. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  3. 2020Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognitioncited by 339
  4. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  5. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 317
  6. 2016Using the Output Embedding to Improve Language Modelscited by 266
  7. 2020Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translationcited by 254
  8. 2019Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparisoncited by 217
  9. 2014Learning Fine-grained Image Similarity with Deep Rankingcited by 189
  10. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
  11. 2020How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Languagecited by 172
  12. 2022End-to-end Generative Pretraining for Multimodal Video Captioningcited by 115
  13. 2022A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translationcited by 80
  14. 2020TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translationcited by 75
  15. 2022Open-Domain Sign Language Translation Learned from Online Videocited by 72
  16. 2018Neural Sign Language Translation based on Human Keypoint Estimationcited by 65
  17. 2022Pretraining with Artificial Language: Studying Transferable Knowledge in Language Modelscited by 28
  18. 2022Keypoint based Sign Language Translation without Glossescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 8; 2026: 12.

31 papers.

Sort:

By area:

  1. 2023YouTube-ASL: A Large-Scale, Open-Domain American Sign Language-English Parallel Corpuscited by 74
  2. 2025Lost in Translation, Found in Context: Sign Language Translation with Contextual Cuescited by 25
  3. 2024C2RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrievalcited by 23
  4. 2024SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scalecited by 18
  5. 2024LLaVA-SLT: Visual Language Tuning for Sign Language Translationcited by 14
  6. 2023Autoregressive Sign Language Production: A Gloss-Free Approach with Discrete Representationscited by 9
  7. 2026Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Fieldcited by 9
  8. 2024Scaling Sign Language Translationcited by 8
  9. 2024CorrNet+: Sign Language Recognition and Translation via Spatial-Temporal Correlationcited by 7
  10. 2024EvSign: Sign Language Recognition and Translation with Streaming Eventscited by 6
  11. 2025Sign Language Translation with Sentence Embedding Supervisioncited by 6
  12. 2025Lost in Translation, Found in Embeddings: Sign Language Translation and Alignmentcited by 6
19 more
  1. 2024A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Productioncited by 5
  2. 2025RVLF: A Reinforcing Vision-Language Framework for Gloss-Free Sign Language Translationcited by 2
  3. 2024Diverse Sign Language Translationcited by 1
  4. 2025SignBind-LLM: Multi-Stage Modality Fusion for Sign Language Translationcited by 1
  5. 2025Text2Sign Diffusion: A Generative Approach for Gloss-Free Sign Language Productioncited by 1
  6. 2024Event Stream-based Sign Language Translation: A High-Definition Benchmark Dataset and A Novel Baselinecited by 0
  7. 2025iLSU-T: an Open Dataset for Uruguayan Sign Language Translationcited by 0
  8. 2025Towards Skeletal and Signer Noise Reduction in Sign Language Production via Quaternion-Based Pose Encoding and Contrastive Learningcited by 0
  9. 2026FEA-SLT: A Gloss-Free End-to-End Framework for Facial-Expression-Aware Sign Language Translationcited by 0
  10. 2026SignDPO: Multi-level Direct Preference Optimisation for Skeleton-based Gloss-free Sign Language Translationcited by 0
  11. 2026Towards Compact Sign Language Translation: Frame Rate and Model Size Trade-offscited by 0
  12. 2026Towards Continuous Sign Language Conversation from Isolated Signscited by 0
  13. 2026Direct Translation between Sign Languagescited by 0
  14. 2026VTaMo: Video-Text Alignment Model for Sign Language Translationcited by 0
  15. 2026Toward Real-Time Sentence-Level Sign Language Translationcited by 0
  16. 2026Q-BridgeNet: A Quantization Network for Cross-Lingual Sign Language Translationcited by 0
  17. 2026SignDino: Self-Supervised Sign Language Representation Learning via Temporal-Axis Self-Distillationcited by 0
  18. 2026OneSign: Unifying Sign Language Understanding Tasks with One Modelcited by 0
  19. 2026SignFLIP: A Unified Model for Sign Language Translation and Generation via Stage-wise Alignment at Scalecited by 0