# Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation

arXiv: https://arxiv.org/abs/1808.09564 · HTML: https://mlsanity.com/paper/1808.09564/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2018.

## Built on by (most cited first, 15 of 15)

- [Scaling Agents via Continual Pre-training](https://mlsanity.com/paper/2509.13310/) (2025)
- [Quiz-Style Question Generation for News Stories](https://mlsanity.com/paper/2102.09094/) (2021)
- [Multiple-Choice Question Generation: Towards an Automated Assessment Framework](https://mlsanity.com/paper/2209.11830/) (2022)
- [Self-Supervised Multimodal Opinion Summarization](https://mlsanity.com/paper/2105.13135/) (2021)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [A First Look: Towards Explainable TextVQA Models via Visual and Textual Explanations](https://mlsanity.com/paper/2105.02626/) (2021)
- [Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References](https://mlsanity.com/paper/2305.15067/) (2023)
- [Pitfalls and Outlooks in Using COMET](https://mlsanity.com/paper/2408.15366/) (2024)
- [Quality and Quantity of Machine Translation References for Automatic Metrics](https://mlsanity.com/paper/2401.01283/) (2024)
- [Learning from Many Voices: Literary MT Using Multi-Reference Human and Synthetic Data](https://mlsanity.com/paper/2412.18707/) (2024)
- [Conciseness: An Overlooked Language Task](https://mlsanity.com/paper/2211.04126/) (2022)
- [DISTO: Evaluating Textual Distractors for Multi-Choice Questions using Negative Sampling based Approach](https://mlsanity.com/paper/2304.04881/) (2023)
- [Diverse Sign Language Translation](https://mlsanity.com/paper/2410.19586/) (2024)
- [Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models](https://mlsanity.com/paper/2605.18504/) (2026)
- [Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders](https://mlsanity.com/paper/2610.17771/) (2026)

## Abstract (arXiv)

Neural text generation, including neural machine translation, image captioning, and summarization, has been quite successful recently. However, during training time, typically only one reference is considered for each example, even though there are often multiple references available, e.g., 4 references in NIST MT evaluations, and 5 references in image captioning data. We first investigate several different ways of utilizing multiple human references during training. But more importantly, we then propose an algorithm to generate exponentially many pseudo-references by first compressing existing human references into lattices and then traversing them to generate new pseudo-references. These approaches lead to substantial improvements over strong baselines in both machine translation (+1.5 BLEU) and image captioning (+3.1 BLEU / +11.7 CIDEr).