mlsanityData to 14 Oct 2026

Document-Level Text Generation with Minimum Bayes Risk Decoding using Optimal Transport

arXiv 2505.23078 ↗First posted 29 May 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Document-level text generation tasks are known to be more difficult than sentence-level text generation tasks as they require the understanding of longer context to generate high-quality texts. In this paper, we investigate the adaption of Minimum Bayes Risk (MBR) decoding for document-level text generation tasks. MBR decoding makes use of a utility function to estimate the output with the highest expected utility from a set of candidate outputs. Although MBR decoding is shown to be effective in a wide range of sentence-level text generation tasks, its performance on document-level text generation tasks is limited as many of the utility functions are designed for evaluating the utility of sentences. To this end, we propose MBR-OT, a variant of MBR decoding using Wasserstein distance to compute the utility of a document using a sentence-level utility function. The experimental result shows that the performance of MBR-OT outperforms that of the standard MBR in document-level machine translation, text simplification, and dense image captioning tasks. Our code is available at https://github.com/jinnaiyuu/mbr-optimal-transport

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 49 of its 79 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

49 of 79 references.

References · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  9. 2020Learning to summarize from human feedbackcited by 2,588
  10. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  11. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  12. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
37 more
  1. 2015Teaching Machines to Read and Comprehendcited by 1,049
  2. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  3. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  4. 2018Computational Optimal Transportcited by 780
  5. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
  6. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  7. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  8. 2017Near-linear time approximation algorithms for optimal transport via Sinkhorn iterationcited by 223
  9. 2024PaliGemma 2: A Family of Versatile VLMs for Transfercited by 207
  10. 2023Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hackingcited by 200
  11. 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
  12. 2018Optimal Transport for structured data with application on graphscited by 172
  13. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
  14. 2024DOCCI: Descriptions of Connected and Contrasting Imagescited by 132
  15. 2022Truncation Sampling as Language Model Desmoothingcited by 123
  16. 2018Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithmcited by 112
  17. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  18. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 65
  19. 2021BlonDe: An Automatic Evaluation Metric for Document-level Machine Translationcited by 58
  20. 2023Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translationcited by 58
  21. 2022Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metriccited by 56
  22. 2024From Pixels to Prose: A Large Dataset of Dense Image Captionscited by 54
  23. 2023Faster Minimum Bayes Risk Decoding with Confidence-based Pruningcited by 34
  24. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33
  25. 2024Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Textscited by 31
  26. 2023Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decodingcited by 28
  27. 2018Optimal Transport: Fast Probabilistic Approximation with Exact Solverscited by 27
  28. 2023BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Trainingcited by 24
  29. 2023Aligning Neural Machine Translation Models: Human Feedback in Training and Inferencecited by 19
  30. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  31. 2024Mitigating Metric Bias in Minimum Bayes Risk Decodingcited by 16
  32. 2024Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decodingcited by 15
  33. 2024Efficient Minimum Bayes Risk Decoding using Low-Rank Matrix Completion Algorithmscited by 13
  34. 2024FullAnno: A Data Engine for Enhancing Image Comprehension of MLLMscited by 12
  35. 2023Model-Based Minimum Bayes Risk Decoding for Text Generationcited by 11
  36. 2024Chasing COMET: Leveraging Minimum Bayes Risk Decoding for Self-Improving Machine Translationcited by 6
  37. 2013Randomized Approximation of the Gram Matrix: Exact Computation and Probabilistic Boundscited by 3

Built on by

Not cited yet.