mlsanityData to 7 Oct 2026

What Do You Get When You Cross Beam Search with Nucleus Sampling?

arXiv 2107.09729 ↗First posted 20 Jul 2021cs.CL cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We combine beam search with the probabilistic pruning technique of nucleus sampling to create two deterministic nucleus search algorithms for natural language generation. The first algorithm, p-exact search, locally prunes the next-token distribution and performs an exact search over the remaining space. The second algorithm, dynamic beam search, shrinks and expands the beam size according to the entropy of the candidate's probability distribution. Despite the probabilistic intuition behind nucleus search, experiments on machine translation and summarization benchmarks show that both algorithms reach the same performance levels as standard beam search.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 18 of its 24 references that are in the map. First posted on arXiv 20 Jul 2021.

Builds on

18 of 24 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  7. 2018Hierarchical Neural Story Generationcited by 1,182
  8. 2017Six Challenges for Neural Machine Translationcited by 334
  9. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  10. 2018Scaling Neural Machine Translationcited by 204
  11. 2019Sparse Sequence-to-Sequence Modelscited by 167
  12. 2018Correcting Length Bias in Neural Machine Translationcited by 119
6 more
  1. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  2. 2020If beam search is the answer, what was the question?cited by 114
  3. 2019compare-mt: A Tool for Holistic Comparison of Language Generation Systemscited by 43
  4. 2019How Decoding Strategies Affect the Verifiability of Generated Textcited by 27
  5. 2021Smoothing and Shrinking the Sparse Seq2Seq Search Spacecited by 11
  6. 2020Why Neural Machine Translation Prefers Empty Outputscited by 6

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 0; 2024: 3; 2025: 1.

7 papers.

  1. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
  2. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  3. 2021Why don't people use character-level machine translation?cited by 27
  4. 2024Non-Exchangeable Conformal Language Generation with Nearest Neighborscited by 21
  5. 2022Predictive Querying for Autoregressive Neural Sequence Modelscited by 5
  6. 2025GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameterscited by 1
  7. 2024Towards Probabilistically-Sound Beam Search with Masked Language Modelscited by 0