mlsanityData to 7 Oct 2026

SumQE: a BERT-based Summary Quality Estimation Model

arXiv 1909.00578 ↗First posted 2 Sep 2019cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

We propose SumQE, a novel Quality Estimation model for summarization based on BERT. The model addresses linguistic quality aspects that are only indirectly captured by content-based approaches to summary evaluation, without involving comparison with human references. SumQE achieves very high correlations with human ratings, outperforming simpler models addressing these linguistic aspects. Predictions of the SumQE model can be used for system development, and to inform users of the quality of automatically produced summaries and other types of generated text.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 11; 2022: 3; 2023: 1; 2024: 1; 2025: 1.

17 papers.

Sort:

  1. 2021MathBERT: A Pre-Trained Model for Mathematical Formula Understandingcited by 58
  2. 2022On the Limitations of Reference-Free Evaluations of Generated Textcited by 52
  3. 2021Finding a Balanced Degree of Automation for Summary Evaluationcited by 43
  4. 2021A Training-free and Reference-free Summarization Evaluation Metric via Centrality-weighted Relevance and Self-referenced Redundancycited by 21
  5. 2021Domain Adaptation with Pre-trained Transformers for Query Focused Abstractive Text Summarizationcited by 21
  6. 2021Play the Shannon Game With Language Models: A Human-Free Approach to Summary Evaluationcited by 12
  7. 2021Decision-Focused Summarizationcited by 11
  8. 2025Reference-free Evaluation Metrics for Text Generation: A Surveycited by 9
  9. 2021How to Evaluate a Summarizer: Study Design and Statistical Analysis for Manual Linguistic Quality Evaluationcited by 7
  10. 2021Diagnostic Captioning: A Surveycited by 6
  11. 2021Estimation of Summary-to-Text Inconsistency by Mismatched Embeddingscited by 5
  12. 2022How to Find Strong Summary Coherence Measures? A Toolbox and a Comparative Study for Summary Coherence Measure Evaluationcited by 4
5 more
  1. 2023A comprehensive review of automatic text summarization techniques: method, data, evaluation and codingcited by 4
  2. 2024Exploring the Correlation between Human and Machine Evaluation of Simultaneous Speech Translationcited by 4
  3. 2021Consistency and Coherence from Points of Contextual Similaritycited by 1
  4. 2021Towards Human-Free Automatic Quality Evaluation of German Summarizationcited by 0
  5. 2022Towards Interpretable Summary Evaluation via Allocation of Contextual Embeddings to Reference Text Topicscited by 0