# A Token-level Contrastive Framework for Sign Language Translation

arXiv: https://arxiv.org/abs/2204.04916 · HTML: https://mlsanity.com/paper/2204.04916/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 11 Apr 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Representation Degeneration Problem in Training Natural Language Generation Models](https://mlsanity.com/paper/1907.12009/) (2019)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [Improving Sign Language Translation with Monolingual Data by Sign Back-Translation](https://mlsanity.com/paper/2105.12397/) (2021)
- [SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization](https://mlsanity.com/paper/2106.01890/) (2021)
- [Contrastive Learning for Many-to-many Multilingual Neural Machine Translation](https://mlsanity.com/paper/2105.09501/) (2021)
- [A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation](https://mlsanity.com/paper/2203.04287/) (2022)
- [TSPNet: Hierarchical Feature Learning via Temporal Semantic Pyramid for Sign Language Translation](https://mlsanity.com/paper/2010.05468/) (2020)
- [Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2002.03187/) (2020)
- [Multi-channel Transformers for Multi-articulatory Sign Language Translation](https://mlsanity.com/paper/2009.00299/) (2020)
- [Including Signed Languages in Natural Language Processing](https://mlsanity.com/paper/2105.05222/) (2021)
- [On Learning Universal Representations Across Languages](https://mlsanity.com/paper/2007.15960/) (2020)
- [Sign Language Translation with Hierarchical Spatio-TemporalGraph Neural Network](https://mlsanity.com/paper/2111.07258/) (2021)
- [PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution](https://mlsanity.com/paper/2107.12600/) (2021)
- [Frequency-Aware Contrastive Learning for Neural Machine Translation](https://mlsanity.com/paper/2112.14484/) (2021)

## Built on by (most cited first, 10 of 10)

- [LLMs are Good Sign Language Translators](https://mlsanity.com/paper/2404.00925/) (2024)
- [Uni-Sign: Toward Unified Sign Language Understanding at Scale](https://mlsanity.com/paper/2501.15187/) (2025)
- [Improving Gloss-free Sign Language Translation by Reducing Representation Density](https://mlsanity.com/paper/2405.14312/) (2024)
- [Leveraging the Power of MLLMs for Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2411.16789/) (2024)
- [A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Production](https://mlsanity.com/paper/2407.02854/) (2024)
- [Linguistics-Vision Monotonic Consistent Network for Sign Language Production](https://mlsanity.com/paper/2412.16944/) (2024)
- [DiffSLT: Enhancing Diversity in Sign Language Translation via Diffusion Model](https://mlsanity.com/paper/2411.17248/) (2024)
- [SignDPO: Multi-level Direct Preference Optimisation for Skeleton-based Gloss-free Sign Language Translation](https://mlsanity.com/paper/2604.18034/) (2026)
- [ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2607.03657/) (2026)
- [VTaMo: Video-Text Alignment Model for Sign Language Translation](https://mlsanity.com/paper/2607.09126/) (2026)

## Abstract (arXiv)

Sign Language Translation (SLT) is a promising technology to bridge the communication gap between the deaf and the hearing people. Recently, researchers have adopted Neural Machine Translation (NMT) methods, which usually require large-scale corpus for training, to achieve SLT. However, the publicly available SLT corpus is very limited, which causes the collapse of the token representations and the inaccuracy of the generated tokens. To alleviate this issue, we propose ConSLT, a novel token-level Contrastive learning framework for Sign Language Translation , which learns effective token representations by incorporating token-level contrastive learning into the SLT decoding process. Concretely, ConSLT treats each token and its counterpart generated by different dropout masks as positive pairs during decoding, and then randomly samples K tokens in the vocabulary that are not in the current sentence to construct negative examples. We conduct comprehensive experiments on two benchmarks (PHOENIX14T and CSL-Daily) for both end-to-end and cascaded settings. The experimental results demonstrate that ConSLT can achieve better translation quality than the strong baselines.