# Improving Back-Translation with Uncertainty-based Confidence Estimation

arXiv: https://arxiv.org/abs/1909.00157 · HTML: https://mlsanity.com/paper/1909.00157/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2019.

## Built on by (most cited first, 30 of 34)

- [Language Model Cascades: Token-level uncertainty and beyond](https://mlsanity.com/paper/2404.10136/) (2024)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks](https://mlsanity.com/paper/2107.07455/) (2021)
- [Calibrating Large Language Models Using Their Generations Only](https://mlsanity.com/paper/2403.05973/) (2024)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Uncertainty-Aware Machine Translation Evaluation](https://mlsanity.com/paper/2109.06352/) (2021)
- [Prevent the Language Model from being Overconfident in Neural Machine Translation](https://mlsanity.com/paper/2105.11098/) (2021)
- [On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Study](https://mlsanity.com/paper/2304.08653/) (2023)
- [A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications](https://mlsanity.com/paper/2111.01968/) (2021)
- [Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training](https://mlsanity.com/paper/2109.02284/) (2021)
- [Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation](https://mlsanity.com/paper/2106.00941/) (2021)
- [On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation](https://mlsanity.com/paper/2110.01811/) (2021)
- [Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction](https://mlsanity.com/paper/2105.04443/) (2021)
- [Calibration of Machine Reading Systems at Scale](https://mlsanity.com/paper/2203.10623/) (2022)
- [Confidence-Aware Scheduled Sampling for Neural Machine Translation](https://mlsanity.com/paper/2107.10427/) (2021)
- [Trust It or Not: Confidence-Guided Automatic Radiology Report Generation](https://mlsanity.com/paper/2106.10887/) (2021)
- [SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers](https://mlsanity.com/paper/2209.06442/) (2022)
- [Active Learning for Abstractive Text Summarization](https://mlsanity.com/paper/2301.03252/) (2023)
- [Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Data](https://mlsanity.com/paper/2305.03827/) (2023)
- [Exploiting Curriculum Learning in Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2109.11177/) (2021)
- [Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU](https://mlsanity.com/paper/2210.12499/) (2022)
- [Self-Modifying State Modeling for Simultaneous Machine Translation](https://mlsanity.com/paper/2406.02237/) (2024)
- [On Synthetic Data for Back Translation](https://mlsanity.com/paper/2310.13675/) (2023)
- [Alternated Training with Synthetic and Authentic Data for Neural Machine Translation](https://mlsanity.com/paper/2106.08582/) (2021)
- [Can We Trust the Performance Evaluation of Uncertainty Estimation Methods in Text Summarization?](https://mlsanity.com/paper/2406.17274/) (2024)
- [RMBR: A Regularized Minimum Bayes Risk Reranking Framework for Machine Translation](https://mlsanity.com/paper/2203.00201/) (2022)
- [Learning Confidence for Transformer-based Neural Machine Translation](https://mlsanity.com/paper/2203.11413/) (2022)
- [Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?](https://mlsanity.com/paper/2211.13865/) (2022)
- [Perception and Semantic Aware Regularization for Sequential Confidence Calibration](https://mlsanity.com/paper/2305.19498/) (2023)
- [CBSiMT: Mitigating Hallucination in Simultaneous Machine Translation with Weighted Prefix-to-Prefix Training](https://mlsanity.com/paper/2311.03672/) (2023)

## Abstract (arXiv)

While back-translation is simple and effective in exploiting abundant monolingual corpora to improve low-resource neural machine translation (NMT), the synthetic bilingual corpora generated by NMT models trained on limited authentic bilingual data are inevitably noisy. In this work, we propose to quantify the confidence of NMT model predictions based on model uncertainty. With word- and sentence-level confidence measures based on uncertainty, it is possible for back-translation to better cope with noise in synthetic bilingual corpora. Experiments on Chinese-English and English-German translation tasks show that uncertainty-based confidence estimation significantly improves the performance of back-translation.