mlsanityData to 7 Oct 2026

Fill in the BLANC: Human-free quality estimation of document summaries

arXiv 2002.09836 ↗First posted 23 Feb 2020cs.CLcited by 80

Not on the map. Why?

Abstract (arXiv)

We present BLANC, a new approach to the automatic estimation of document summary quality. Our goal is to measure the functional performance of a summary with an objective, reproducible, and fully automated method. Our approach achieves this by measuring the performance boost gained by a pre-trained language model with access to a document summary while carrying out its language understanding task on the document's text. We present evidence that BLANC scores have as good correlation with human evaluations as do the ROUGE family of summary quality measurements. And unlike ROUGE, the BLANC method does not require human-written reference summaries, allowing for fully human-free summary quality estimation.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 12; 2023: 16; 2024: 19; 2025: 17; 2026: 6.

80 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  3. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  4. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  5. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
  6. 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 146
  7. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 141
  8. 2023BooookScore: A systematic exploration of book-length summarization in the era of LLMscited by 137
  9. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 73
  10. 2022On the Limitations of Reference-Free Evaluations of Generated Textcited by 52
  11. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  12. 2021Perturbation CheckLists for Evaluating NLG Evaluation Metricscited by 49
38 more
  1. 2021Finding a Balanced Degree of Automation for Summary Evaluationcited by 43
  2. 2023Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theorycited by 41
  3. 2021What's in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarizationcited by 35
  4. 2024Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluationcited by 32
  5. 2023Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluationscited by 18
  6. 2023Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Surveycited by 18
  7. 2024Legal Judgment Reimagined: PredEx and the Rise of Intelligent AI Interpretation in Indian Courtscited by 15
  8. 2023OpinSummEval: Revisiting Automated Evaluation for Opinion Summarizationcited by 13
  9. 2021Play the Shannon Game With Language Models: A Human-Free Approach to Summary Evaluationcited by 12
  10. 2021Robust Deep Reinforcement Learning for Extractive Legal Summarizationcited by 10
  11. 2022Abstractive Meeting Summarization: A Surveycited by 9
  12. 2022Questioning the Validity of Summarization Datasets and Improving Their Factual Consistencycited by 9
  13. 2024LOCOST: State-Space Models for Long Document Abstractive Summarizationcited by 8
  14. 2025HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Servingcited by 8
  15. 2023Neural Natural Language Processing for Long Texts: A Survey on Classification and Summarizationcited by 7
  16. 2024textttCOSMIC: Mutual Information for Task-Agnostic Summarization Evaluationcited by 6
  17. 2024Information-Theoretic Distillation for Reference-less Summarizationcited by 6
  18. 2025SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?cited by 6
  19. 2025Identifying Reliable Evaluation Metrics for Scientific Text Revisioncited by 6
  20. 2021Estimation of Summary-to-Text Inconsistency by Mismatched Embeddingscited by 5
  21. 2023Hybrid Long Document Summarization using C2F-FAR and ChatGPT: A Practical Studycited by 5
  22. 2024Knowledge-Centric Templatic Views of Documentscited by 5
  23. 2025Summarizing Speech: A Comprehensive Surveycited by 5
  24. 2025NyayaRAG: Realistic Legal Judgment Prediction with RAG under the Indian Common Law Systemcited by 5
  25. 2023A comprehensive review of automatic text summarization techniques: method, data, evaluation and codingcited by 4
  26. 2023Evaluating OpenAI's Whisper ASR for Punctuation Prediction and Topic Modeling of life histories of the Museum of the Personcited by 4
  27. 2024NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision Analysiscited by 4
  28. 2023UMSE: Unified Multi-scenario Summarization Evaluationcited by 3
  29. 2023Correction of Errors in Preference Ratings from Automated Metrics for Text Generationcited by 3
  30. 2024REFeREE: A REference-FREE Model-Based Metric for Text Simplificationcited by 3
  31. 2024PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detectioncited by 3
  32. 2024Speech vs. Transcript: Does It Matter for Human Annotators in Speech Summarization?cited by 3
  33. 2024Summarizing long regulatory documents with a multi-step pipelinecited by 3
  34. 2024Large Language Models Meet Graph Neural Networks: A Perspective of Graph Miningcited by 3
  35. 2022Repro: An Open-Source Library for Improving the Reproducibility and Usability of Publicly Available Research Codecited by 2
  36. 2022DocAsRef: An Empirical Study on Repurposing Reference-Based Summary Quality Metrics Reference-Freelycited by 2
  37. 2023Mapping the Challenges of HCI: An Application and Evaluation of ChatGPT for Mining Insights at Scalecited by 2
  38. 2024Favi-Score: A Measure for Favoritism in Automated Preference Ratings for Generative AI Evaluationcited by 2

50 most cited of 80 All 80 (JSON)