# The Neural Noisy Channel

arXiv: https://arxiv.org/abs/1611.02554 · HTML: https://mlsanity.com/paper/1611.02554/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2016.

## Built on by (most cited first, 19 of 19)

- [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://mlsanity.com/paper/2108.04106/) (2021)
- [OpenICL: An Open-Source Framework for In-context Learning](https://mlsanity.com/paper/2303.02913/) (2023)
- [Quality-Aware Decoding for Neural Machine Translation](https://mlsanity.com/paper/2205.00978/) (2022)
- [Pretraining the Noisy Channel Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2103.10518/) (2021)
- [Assessing Reference-Free Peer Evaluation for Machine Translation](https://mlsanity.com/paper/2104.05146/) (2021)
- [Simpson's Paradox and the Accuracy-Fluency Tradeoff in Translation](https://mlsanity.com/paper/2402.12690/) (2024)
- [A Comparison of Approaches to Document-level Machine Translation](https://mlsanity.com/paper/2101.11040/) (2021)
- [A study of latent monotonic attention variants](https://mlsanity.com/paper/2103.16710/) (2021)
- [Adapting Document-Grounded Dialog Systems to Spoken Conversations using Data Augmentation and a Noisy Channel Model](https://mlsanity.com/paper/2112.08844/) (2021)
- [Amortized Noisy Channel Neural Machine Translation](https://mlsanity.com/paper/2112.08670/) (2021)
- [Monotonic segmental attention for automatic speech recognition](https://mlsanity.com/paper/2210.14742/) (2022)
- [Task-oriented Document-Grounded Dialog Systems by HLTPR@RWTH for DSTC9 and DSTC10](https://mlsanity.com/paper/2304.07101/) (2023)
- [Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model](https://mlsanity.com/paper/2210.17418/) (2022)
- [Context-Aware Machine Translation with Source Coreference Explanation](https://mlsanity.com/paper/2404.19505/) (2024)
- [LEDOM: Reverse Language Model](https://mlsanity.com/paper/2507.01335/) (2025)
- [Case-Based Decision-Theoretic Decoding with Quality Memories](https://mlsanity.com/paper/2509.12677/) (2025)
- [Enabling arbitrary translation objectives with Adaptive Tree Search](https://mlsanity.com/paper/2202.11444/) (2022)
- [Noisy Channel for Automatic Text Simplification](https://mlsanity.com/paper/2211.03152/) (2022)
- [Latent Optimal Paths by Gumbel Propagation for Variational Bayesian Dynamic Programming](https://mlsanity.com/paper/2306.02568/) (2023)

## Abstract (arXiv)

We formulate sequence to sequence transduction as a noisy channel decoding problem and use recurrent neural networks to parameterise the source and channel models. Unlike direct models which can suffer from explaining-away effects during training, noisy channel models must produce outputs that explain their inputs, and their component models can be trained with not only paired training samples but also unpaired samples from the marginal output distribution. Using a latent variable to control how much of the conditioning sequence the channel model needs to read in order to generate a subsequent symbol, we obtain a tractable and effective beam search decoder. Experimental results on abstractive sentence summarisation, morphological inflection, and machine translation show that noisy channel models outperform direct models, and that they significantly benefit from increased amounts of unpaired output data that direct models cannot easily use.