# Sensitivity of BLANC to human-scored qualities of text summaries

arXiv: https://arxiv.org/abs/2010.06716 · HTML: https://mlsanity.com/paper/2010.06716/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 6 of 6)

- [LightTag: Text Annotation Platform](https://mlsanity.com/paper/2109.02320/) (2021)
- [Play the Shannon Game With Language Models: A Human-Free Approach to Summary Evaluation](https://mlsanity.com/paper/2103.10918/) (2021)
- [Estimation of Summary-to-Text Inconsistency by Mismatched Embeddings](https://mlsanity.com/paper/2104.05156/) (2021)
- [Summarizing long regulatory documents with a multi-step pipeline](https://mlsanity.com/paper/2408.09777/) (2024)
- [Consistency and Coherence from Points of Contextual Similarity](https://mlsanity.com/paper/2112.11638/) (2021)
- [Towards Human-Free Automatic Quality Evaluation of German Summarization](https://mlsanity.com/paper/2105.06027/) (2021)

## Abstract (arXiv)

We explore the sensitivity of a document summary quality estimator, BLANC, to human assessment of qualities for the same summaries. In our human evaluations, we distinguish five summary qualities, defined by how fluent, understandable, informative, compact, and factually correct the summary is. We make the case for optimal BLANC parameters, at which the BLANC sensitivity to almost all of summary qualities is about as good as the sensitivity of a human annotator.