# Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study

arXiv: https://arxiv.org/abs/1909.06618 · HTML: https://mlsanity.com/paper/1909.06618/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2019.

## Built on by (most cited first, 4 of 4)

- [Influence Scores at Scale for Efficient Language Data Sampling](https://mlsanity.com/paper/2311.16298/) (2023)
- [Topic Segmentation of Research Article Collections](https://mlsanity.com/paper/2205.11249/) (2022)
- [Differentially-private text generation degrades output language quality](https://mlsanity.com/paper/2509.11176/) (2025)
- [AlbNews: A Corpus of Headlines for Topic Modeling in Albanian](https://mlsanity.com/paper/2402.04028/) (2024)

## Abstract (arXiv)

Using data-driven models for solving text summarization or similar tasks has become very common in the last years. Yet most of the studies report basic accuracy scores only, and nothing is known about the ability of the proposed models to improve when trained on more data. In this paper, we define and propose three data efficiency metrics: data score efficiency, data time deficiency and overall data efficiency. We also propose a simple scheme that uses those metrics and apply it for a more comprehensive evaluation of popular methods on text summarization and title generation tasks. For the latter task, we process and release a huge collection of 35 million abstract-title pairs from scientific articles. Our results reveal that among the tested models, the Transformer is the most efficient on both tasks.