# Stochastic Transformer Networks with Linear Competing Units: Application to end-to-end SL Translation

arXiv: https://arxiv.org/abs/2109.13318 · HTML: https://mlsanity.com/paper/2109.13318/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation](https://mlsanity.com/paper/2003.13830/) (2020)
- [Bayesian Compression for Deep Learning](https://mlsanity.com/paper/1705.08665/) (2017)
- [Spatial-Temporal Multi-Cue Network for Continuous Sign Language Recognition](https://mlsanity.com/paper/2002.03187/) (2020)
- [Multi-channel Transformers for Multi-articulatory Sign Language Translation](https://mlsanity.com/paper/2009.00299/) (2020)
- [Quantitative Survey of the State of the Art in Sign Language Recognition](https://mlsanity.com/paper/2008.09918/) (2020)
- [Joey NMT: A Minimalist NMT Toolkit for Novices](https://mlsanity.com/paper/1907.12484/) (2019)
- [Neural Sign Language Translation by Learning Tokenization](https://mlsanity.com/paper/2002.00479/) (2020)
- [Nonparametric Bayesian Deep Networks with Local Competition](https://mlsanity.com/paper/1805.07624/) (2018)
- [Variational Bayesian Sequence-to-Sequence Networks for Memory-Efficient Sign Language Translation](https://mlsanity.com/paper/2102.06143/) (2021)

## Built on by (most cited first, 30 of 35)

- [TransDreamer: Reinforcement Learning with Transformer World Models](https://mlsanity.com/paper/2202.09481/) (2022)
- [Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining](https://mlsanity.com/paper/2307.14768/) (2023)
- [LLMs are Good Sign Language Translators](https://mlsanity.com/paper/2404.00925/) (2024)
- [Gloss Attention for Gloss-free Sign Language Translation](https://mlsanity.com/paper/2307.07361/) (2023)
- [Sign Language Translation from Instructional Videos](https://mlsanity.com/paper/2304.06371/) (2023)
- [Machine Translation from Signed to Spoken Languages: State of the Art and Challenges](https://mlsanity.com/paper/2202.03086/) (2022)
- [JWSign: A Highly Multilingual Corpus of Bible Translations for more Diversity in Sign Language Processing](https://mlsanity.com/paper/2311.10174/) (2023)
- [Leveraging the Power of MLLMs for Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2411.16789/) (2024)
- [Stochastic Local Winner-Takes-All Networks Enable Profound Adversarial Robustness](https://mlsanity.com/paper/2112.02671/) (2021)
- [An Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMs](https://mlsanity.com/paper/2408.10593/) (2024)
- [DISCOVER: Making Vision Networks Interpretable via Competition and Dissection](https://mlsanity.com/paper/2310.04929/) (2023)
- [Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jets](https://mlsanity.com/paper/2310.16741/) (2023)
- [Stochastic Deep Networks with Linear Competing Units for Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2208.01573/) (2022)
- [Beyond Gloss: A Hand-Centric Framework for Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2507.23575/) (2025)
- [A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Dataset](https://mlsanity.com/paper/2310.04753/) (2023)
- [iSign: A Benchmark for Indian Sign Language Processing](https://mlsanity.com/paper/2407.05404/) (2024)
- [Hierarchical Feature Alignment for Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2507.06732/) (2025)
- [A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Production](https://mlsanity.com/paper/2407.02854/) (2024)
- [SEDS: Semantically Enhanced Dual-Stream Encoder for Sign Language Retrieval](https://mlsanity.com/paper/2407.16394/) (2024)
- [SCOPE: Sign Language Contextual Processing with Embedding from LLMs](https://mlsanity.com/paper/2409.01073/) (2024)
- [Competing Mutual Information Constraints with Stochastic Competition-based Activations for Learning Diversified Representations](https://mlsanity.com/paper/2201.03624/) (2022)
- [Tackling Low-Resourced Sign Language Translation: UPC at WMT-SLT 22](https://mlsanity.com/paper/2212.01140/) (2022)
- [Transformers with Stochastic Competition for Tabular Data Modelling](https://mlsanity.com/paper/2407.13238/) (2024)
- [Signformer is all you need: Towards Edge AI for Sign Language](https://mlsanity.com/paper/2411.12901/) (2024)
- [Neural Sign Reenactor: Deep Photorealistic Sign Language Retargeting](https://mlsanity.com/paper/2209.01470/) (2022)
- [ISLTranslate: Dataset for Translating Indian Sign Language](https://mlsanity.com/paper/2307.05440/) (2023)
- [Continual Deep Learning on the Edge via Stochastic Local Competition among Subnetworks](https://mlsanity.com/paper/2407.10758/) (2024)
- [DiffSLT: Enhancing Diversity in Sign Language Translation via Diffusion Model](https://mlsanity.com/paper/2411.17248/) (2024)
- [Think in Latent Thoughts: A New Paradigm for Gloss-Free Sign Language Translation](https://mlsanity.com/paper/2604.15301/) (2026)
- [Exploring Attention Mechanisms in Integration of Multi-Modal Information for Sign Language Recognition and Translation](https://mlsanity.com/paper/2309.01860/) (2023)

## Abstract (arXiv)

Automating sign language translation (SLT) is a challenging real world application. Despite its societal importance, though, research progress in the field remains rather poor. Crucially, existing methods that yield viable performance necessitate the availability of laborious to obtain gloss sequence groundtruth. In this paper, we attenuate this need, by introducing an end-to-end SLT model that does not entail explicit use of glosses; the model only needs text groundtruth. This is in stark contrast to existing end-to-end models that use gloss sequence groundtruth, either in the form of a modality that is recognized at an intermediate model stage, or in the form of a parallel output process, jointly trained with the SLT model. Our approach constitutes a Transformer network with a novel type of layers that combines: (i) local winner-takes-all (LWTA) layers with stochastic winner sampling, instead of conventional ReLU layers, (ii) stochastic weights with posterior distributions estimated via variational inference, and (iii) a weight compression technique at inference time that exploits estimated posterior variance to perform massive, almost lossless compression. We demonstrate that our approach can reach the currently best reported BLEU-4 score on the PHOENIX 2014T benchmark, but without making use of glosses for model training, and with a memory footprint reduced by more than 70%.