# Fairness for Whom? Understanding the Reader's Perception of Fairness in Text Summarization

arXiv: https://arxiv.org/abs/2101.12406 · HTML: https://mlsanity.com/paper/2101.12406/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 19 references that are in the map.
- First posted on arXiv 29 Jan 2021.

## Builds on (most cited first, 6 of 6)

- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [FairRec: Two-Sided Fairness for Personalized Recommendations in Two-Sided Platforms](https://mlsanity.com/paper/2002.10764/) (2020)
- [ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks](https://mlsanity.com/paper/1803.01937/) (2018)
- [Text Summarization Techniques: A Brief Survey](https://mlsanity.com/paper/1707.02268/) (2017)
- [Read what you need: Controllable Aspect-based Opinion Summarization of Tourist Reviews](https://mlsanity.com/paper/2006.04660/) (2020)
- [Trump vs. Hillary: What went Viral during the 2016 US Presidential Election](https://mlsanity.com/paper/1707.03375/) (2017)

## Built on by (most cited first, 8 of 8)

- [Fair Abstractive Summarization of Diverse Perspectives](https://mlsanity.com/paper/2311.07884/) (2023)
- [Examining Bias in Opinion Summarisation Through the Perspective of Opinion Diversity](https://mlsanity.com/paper/2306.04424/) (2023)
- [Methodological reflections for AI alignment research using human feedback](https://mlsanity.com/paper/2301.06859/) (2022)
- [Summarizing Speech: A Comprehensive Survey](https://mlsanity.com/paper/2504.08024/) (2025)
- [Responsible AI Considerations in Text Summarization Research: A Review of Current Practices](https://mlsanity.com/paper/2311.11103/) (2023)
- [Less Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinions](https://mlsanity.com/paper/2508.17610/) (2025)
- [Coverage-based Fairness in Multi-document Summarization](https://mlsanity.com/paper/2412.08795/) (2024)
- [Faithful Summarisation under Disagreement via Belief-Level Aggregation](https://mlsanity.com/paper/2601.04889/) (2026)

## Abstract (arXiv)

With the surge in user-generated textual information, there has been a recent increase in the use of summarization algorithms for providing an overview of the extensive content. Traditional metrics for evaluation of these algorithms (e.g. ROUGE scores) rely on matching algorithmic summaries to human-generated ones. However, it has been shown that when the textual contents are heterogeneous, e.g., when they come from different socially salient groups, most existing summarization algorithms represent the social groups very differently compared to their distribution in the original data. To mitigate such adverse impacts, some fairness-preserving summarization algorithms have also been proposed. All of these studies have considered normative notions of fairness from the perspective of writers of the contents, neglecting the readers' perceptions of the underlying fairness notions. To bridge this gap, in this work, we study the interplay between the fairness notions and how readers perceive them in textual summaries. Through our experiments, we show that reader's perception of fairness is often context-sensitive. Moreover, standard ROUGE evaluation metrics are unable to quantify the perceived (un)fairness of the summaries. To this end, we propose a human-in-the-loop metric and an automated graph-based methodology to quantify the perceived bias in textual summaries. We demonstrate their utility by quantifying the (un)fairness of several summaries of heterogeneous socio-political microblog datasets.