mlsanityData to 7 Oct 2026

Improving Minimum Bayes Risk Decoding with Multi-Prompt

arXiv 2407.15343 ↗First posted 22 Jul 2024cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

While instruction fine-tuned LLMs are effective text generators, sensitivity to prompt construction makes performance unstable and sub-optimal in practice. Relying on a single "best" prompt cannot capture all differing approaches to a generation problem. Using this observation, we propose multi-prompt decoding, where many candidate generations are decoded from a prompt bank at inference-time. To ensemble candidates, we use Minimum Bayes Risk (MBR) decoding, which selects a final output using a trained value metric. We show multi-prompt improves MBR across a comprehensive set of conditional generation tasks, and show this is a result of estimating a more diverse and higher quality candidate space than that of a single prompt. Further experiments confirm multi-prompt improves generation across tasks, models and metrics.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 58 of its 82 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

58 of 82 references.

References · 58

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2019The Curious Case of Neural Text Degenerationcited by 2,881
38 more
  1. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  6. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  7. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  8. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  9. 2023StarCoder: may the source be with you!cited by 790
  10. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  11. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  12. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  13. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  14. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  15. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  16. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  17. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  18. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  19. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  20. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
  21. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  22. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  23. 2022In-context Examples Selection for Machine Translationcited by 172
  24. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  25. 2022Natural Language to Code Translation with Executioncited by 122
  26. 2020Neural CRF Model for Sentence Alignment in Text Simplificationcited by 106
  27. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  28. 2020ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformationscited by 101
  29. 2023Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Taskcited by 88
  30. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 86
  31. 2022Coder Reviewer Reranking for Code Generationcited by 83
  32. 2021Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Promptscited by 64
  33. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 63
  34. 2023Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translationcited by 57
  35. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  36. 2022Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decodingcited by 54
  37. 2022LENS: A Learnable Evaluation Metric for Text Simplificationcited by 54
  38. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 49

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  2. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMscited by 162
  3. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  4. 2025Evaluation of Best-of-N Sampling Strategies for Language Model Alignmentcited by 38
  5. 2026CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decodingcited by 13
  6. 2024Diversity Explains Inference Scaling Laws: Through a Case Study of Minimum Bayes Risk Decodingcited by 8
  7. 2024M-Ped: Multi-Prompt Ensemble Decoding for Large Language Modelscited by 3
  8. 2025Theoretical Guarantees for Minimum Bayes Risk Decodingcited by 3
  9. 2025Case-Based Decision-Theoretic Decoding with Quality Memoriescited by 1
  10. 2026VeriSpace: Spatially Grounded Action Verification for Vision-Language-Action Modelscited by 1
  11. 2025Minimum Bayes Risk Decoding for Error Span Detection in Reference-Free Automatic Machine Translation Evaluationcited by 0
  12. 2026Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NERcited by 0
1 more
  1. 2026Enhancing Small Language Models for Power Outage Report Generation via Minimum Risk Trainingcited by 0