# Understanding the Impact of Text Highlighting in Crowdsourcing Tasks

arXiv: https://arxiv.org/abs/1909.02780 · HTML: https://mlsanity.com/paper/1909.02780/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2019.

## Built on by (most cited first, 3 of 3)

- [HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputs](https://mlsanity.com/paper/2503.02003/) (2025)
- [Word-level Text Highlighting of Medical Texts for Telehealth Services](https://mlsanity.com/paper/2105.10400/) (2021)
- [Studying and Recommending Information Highlighting in Stack Overflow Answers](https://mlsanity.com/paper/2401.01472/) (2024)

## Abstract (arXiv)

Text classification is one of the most common goals of machine learning (ML) projects, and also one of the most frequent human intelligence tasks in crowdsourcing platforms. ML has mixed success in such tasks depending on the nature of the problem, while crowd-based classification has proven to be surprisingly effective, but can be expensive. Recently, hybrid text classification algorithms, combining human computation and machine learning, have been proposed to improve accuracy and reduce costs. One way to do so is to have ML highlight or emphasize portions of text that it believes to be more relevant to the decision. Humans can then rely only on this text or read the entire text if the highlighted information is insufficient. In this paper, we investigate if and under what conditions highlighting selected parts of the text can (or cannot) improve classification cost and/or accuracy, and in general how it affects the process and outcome of the human intelligence tasks. We study this through a series of crowdsourcing experiments running over different datasets and with task designs imposing different cognitive demands. Our findings suggest that highlighting is effective in reducing classification effort but does not improve accuracy - and in fact, low-quality highlighting can decrease it.