# Confidence through Attention

arXiv: https://arxiv.org/abs/1710.03743 · HTML: https://mlsanity.com/paper/1710.03743/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2017.

## Built on by (most cited first, 8 of 8)

- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [Sparse Attention with Linear Units](https://mlsanity.com/paper/2104.07012/) (2021)
- [Translation Error Detection as Rationale Extraction](https://mlsanity.com/paper/2108.12197/) (2021)
- [Revisiting Negation in Neural Machine Translation](https://mlsanity.com/paper/2107.12203/) (2021)
- [Self-Evaluation of Large Language Model based on Glass-box Features](https://mlsanity.com/paper/2403.04222/) (2024)
- [Quality Estimation with k-nearest Neighbors and Automatic Evaluation for Model-specific Quality Estimation](https://mlsanity.com/paper/2404.18031/) (2024)
- [Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehension](https://mlsanity.com/paper/2311.06758/) (2023)
- [Using Source-Side Confidence Estimation for Reliable Translation into Unfamiliar Languages](https://mlsanity.com/paper/2503.23305/) (2025)

## Abstract (arXiv)

Attention distributions of the generated translations are a useful bi-product of attention-based recurrent neural network translation models and can be treated as soft alignments between the input and output tokens. In this work, we use attention distributions as a confidence metric for output translations. We present two strategies of using the attention distributions: filtering out bad translations from a large back-translated corpus, and selecting the best translation in a hybrid setup of two different translation systems. While manual evaluation indicated only a weak correlation between our confidence score and human judgments, the use-cases showed improvements of up to 2.22 BLEU points for filtering and 0.99 points for hybrid translation, tested on English<->German and English<->Latvian translation.