mlsanityData to 7 Oct 2026

deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets

arXiv 1506.06863 ↗First posted 23 Jun 2015cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

We introduce Discriminative BLEU (deltaBLEU), a novel metric for intrinsic evaluation of generated text in tasks that admit a diverse range of possible outputs. Reference strings are scored for quality by human raters on a scale of [-1, +1] to weight multi-reference BLEU. In tasks involving generation of conversational responses, deltaBLEU correlates reasonably with human judgments and outperforms sentence-level and IBM BLEU in terms of both Spearman's rho and Kendall's tau.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 10; 2022: 5; 2023: 1; 2024: 2; 2025: 3.

21 papers.

Sort:

By area:

  1. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  2. 2021A Comprehensive Assessment of Dialog Evaluation Metricscited by 90
  3. 2021Shades of BLEU, Flavours of Success: The Case of MultiWOZcited by 48
  4. 2021InfoLM: A New Metric to Evaluate Summarization & Data2Text Generationcited by 33
  5. 2022Why is constrained neural language generation particularly challenging?cited by 19
  6. 2021Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approachcited by 16
  7. 2025MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question Answeringcited by 8
  8. 2021Language Model Augmented Relevance Scorecited by 7
  9. 2021Meta-evaluation of Conversational Search Evaluation Metricscited by 6
  10. 2021Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentationcited by 6
  11. 2022Evaluation of Question Answering Systems: Complexity of judging a natural languagecited by 5
  12. 2022Learning to Improve Code Efficiencycited by 4
9 more
  1. 2025Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQAcited by 3
  2. 2021How to Evaluate Your Dialogue Models: A Review of Approachescited by 2
  3. 2025Calibrating Translation Decoding with Quality Estimation on LLMscited by 2
  4. 2021On the Use of Linguistic Features for the Evaluation of Generative Dialogue Systemscited by 1
  5. 2023How to Choose How to Choose Your Chatbot: A Massively Multi-System MultiReference Data Set for Dialog Metric Evaluationcited by 1
  6. 2022Should we tweet this? Generative response modeling for predicting reception of public health messaging on Twittercited by 0
  7. 2022Improving a sequence-to-sequence nlp model using a reinforcement learning policy algorithmcited by 0
  8. 2024Rethinking Response Evaluation from Interlocutor's Eye for Open-Domain Dialogue Systemscited by 0
  9. 2024WangLab at MEDIQA-M3G 2024: Multimodal Medical Answer Generation using Large Language Modelscited by 0