# Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers

arXiv: https://arxiv.org/abs/2006.08344 · HTML: https://mlsanity.com/paper/2006.08344/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2020.

## Built on by (most cited first, 30 of 31)

- [Out-of-Distribution Detection and Selective Generation for Conditional Language Models](https://mlsanity.com/paper/2209.15558/) (2022)
- [Language Model Cascades: Token-level uncertainty and beyond](https://mlsanity.com/paper/2404.10136/) (2024)
- [Calibrating Large Language Models Using Their Generations Only](https://mlsanity.com/paper/2403.05973/) (2024)
- [Uncertainty in Natural Language Generation: From Theory to Applications](https://mlsanity.com/paper/2307.15703/) (2023)
- [MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMs](https://mlsanity.com/paper/2402.11756/) (2024)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity](https://mlsanity.com/paper/2210.15452/) (2022)
- [Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data](https://mlsanity.com/paper/2212.09171/) (2022)
- [Non-Exchangeable Conformal Language Generation with Nearest Neighbors](https://mlsanity.com/paper/2402.00707/) (2024)
- [Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training](https://mlsanity.com/paper/2109.02284/) (2021)
- [Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning](https://mlsanity.com/paper/2405.14039/) (2024)
- [Out of Distribution Data Detection Using Dropout Bayesian Neural Networks](https://mlsanity.com/paper/2202.08985/) (2022)
- [Should We Trust This Summary? Bayesian Abstractive Summarization to The Rescue](https://mlsanity.com/paper/2105.10155/) (2021)
- [Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector](https://mlsanity.com/paper/2404.08679/) (2024)
- [Trust It or Not: Confidence-Guided Automatic Radiology Report Generation](https://mlsanity.com/paper/2106.10887/) (2021)
- [Active Learning for Abstractive Text Summarization](https://mlsanity.com/paper/2301.03252/) (2023)
- [Semantic Uncertainty Quantification of Hallucinations in LLMs: A Quantum Tensor Network Based Method](https://mlsanity.com/paper/2601.20026/) (2026)
- [Bayesian Active Summarization](https://mlsanity.com/paper/2110.04480/) (2021)
- [Combining Data Generation and Active Learning for Low-Resource Question Answering](https://mlsanity.com/paper/2211.14880/) (2022)
- [aiXamine: Simplified LLM Safety and Security](https://mlsanity.com/paper/2504.14985/) (2025)
- [Improving the Calibration of Confidence Scores in Text Generation Using the Output Distribution's Characteristics](https://mlsanity.com/paper/2506.00637/) (2025)
- [Active Learning for Natural Language Generation](https://mlsanity.com/paper/2305.15040/) (2023)
- [Learning Confidence for Transformer-based Neural Machine Translation](https://mlsanity.com/paper/2203.11413/) (2022)
- [Enhancing RAG with Active Learning on Conversation Records: Reject Incapables and Answer Capables](https://mlsanity.com/paper/2502.09073/) (2025)
- [DA3: A Distribution-Aware Adversarial Attack against Language Models](https://mlsanity.com/paper/2311.08598/) (2023)
- [AP-OOD: Attention Pooling for Out-of-Distribution Detection](https://mlsanity.com/paper/2602.06031/) (2026)
- [Variational Neurons in Transformers for Language Modeling](https://mlsanity.com/paper/2603.28219/) (2026)
- [ATGen: A Framework for Active Text Generation](https://mlsanity.com/paper/2506.23342/) (2025)
- [ESG-Bench: Benchmarking Long-Context ESG Reports for Hallucination Mitigation](https://mlsanity.com/paper/2603.13154/) (2026)
- [Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning](https://mlsanity.com/paper/2604.08974/) (2026)

## Abstract (arXiv)

We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models. For this we develop a new measure of uncertainty designed specifically for long sequences of discrete random variables -- i.e. words in the output sentence. Our new measure of uncertainty solves a major intractability in the naive application of existing approaches on long sentences. We use our new measure on a Transformer model trained with dropout approximate inference. On the task of German-English translation using WMT13 and Europarl, we show that with dropout uncertainty our measure is able to identify when Dutch source sentences, sentences which use the same word types as German, are given to the model instead of German.