# Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model

arXiv: https://arxiv.org/abs/1605.03835 · HTML: https://mlsanity.com/paper/1605.03835/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2016.

## Built on by (most cited first, 18 of 18)

- [Jointly Reinforcing Diversity and Quality in Language Model Generations](https://mlsanity.com/paper/2509.02534/) (2025)
- [Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models](https://mlsanity.com/paper/2210.15458/) (2022)
- [ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning](https://mlsanity.com/paper/2105.12995/) (2021)
- [Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Survey](https://mlsanity.com/paper/2311.09212/) (2023)
- [QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translation](https://mlsanity.com/paper/2406.00049/) (2024)
- [Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2401.05054/) (2024)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Priority Sampling of Large Language Models for Compilers](https://mlsanity.com/paper/2402.18734/) (2024)
- [ParaLS: Lexical Substitution via Pretrained Paraphraser](https://mlsanity.com/paper/2305.08146/) (2023)
- [Enhancing In-Context Learning via Implicit Demonstration Augmentation](https://mlsanity.com/paper/2407.00100/) (2024)
- [A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models](https://mlsanity.com/paper/2405.13019/) (2024)
- [Geometry of Knowledge Allows Extending Diversity Boundaries of Large Language Models](https://mlsanity.com/paper/2507.13874/) (2025)
- [Generating Diverse Translation with Perturbed kNN-MT](https://mlsanity.com/paper/2402.09344/) (2024)
- [Inducing Sustained Creativity and Diversity in Large Language Models](https://mlsanity.com/paper/2603.19519/) (2026)
- [Optimizing Non-Autoregressive Transformers with Contrastive Learning](https://mlsanity.com/paper/2305.13667/) (2023)
- [Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models](https://mlsanity.com/paper/2505.09062/) (2025)
- [Exploring Representational Disparities Between Multilingual and Bilingual Translation Models](https://mlsanity.com/paper/2305.14230/) (2023)
- [AMLNet: Adversarial Mutual Learning Neural Network for Non-AutoRegressive Multi-Horizon Time Series Forecasting](https://mlsanity.com/paper/2310.19289/) (2023)

## Abstract (arXiv)

Recent advances in conditional recurrent language modelling have mainly focused on network architectures (e.g., attention mechanism), learning algorithms (e.g., scheduled sampling and sequence-level training) and novel applications (e.g., image/video description generation, speech recognition, etc.) On the other hand, we notice that decoding algorithms/strategies have not been investigated as much, and it has become standard to use greedy or beam search. In this paper, we propose a novel decoding strategy motivated by an earlier observation that nonlinear hidden layers of a deep neural network stretch the data manifold. The proposed strategy is embarrassingly parallelizable without any communication overhead, while improving an existing decoding algorithm. We extensively evaluate it with attention-based neural machine translation on the task of En->Cz translation.