# Navigating the Thin Line: Examining User Behavior in Search to Detect Engagement and Backfire Effects

arXiv: https://arxiv.org/abs/2401.11201 · HTML: https://mlsanity.com/paper/2401.11201/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 20 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [FA*IR: A Fair Top-k Ranking Algorithm](https://mlsanity.com/paper/1706.06368/) (2017)
- [Seeing Things from a Different Angle: Discovering Diverse Perspectives about Claims](https://mlsanity.com/paper/1906.03538/) (2019)
- [Quantifying the Impact of User Attention on Fair Group Representation in Ranked Lists](https://mlsanity.com/paper/1901.10437/) (2019)
- [Evaluation Metrics for Measuring Bias in Search Engine Results](https://mlsanity.com/paper/2210.10517/) (2022)
- [Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics](https://mlsanity.com/paper/2010.14531/) (2020)
- [RADio -- Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations](https://mlsanity.com/paper/2209.13520/) (2022)
- [Evaluating Fairness in Argument Retrieval](https://mlsanity.com/paper/2108.10442/) (2021)
- [Helping users discover perspectives: Enhancing opinion mining with joint topic models](https://mlsanity.com/paper/2010.12505/) (2020)
- [Explaining Search Result Stances to Opinionated People](https://mlsanity.com/paper/2309.08460/) (2023)

## Abstract (arXiv)

Opinionated users often seek information that aligns with their preexisting beliefs while dismissing contradictory evidence due to confirmation bias. This conduct hinders their ability to consider alternative stances when searching the web. Despite this, few studies have analyzed how the diversification of search results on disputed topics influences the search behavior of highly opinionated users. To this end, we present a preregistered user study (n = 257) investigating whether different levels (low and high) of bias metrics and search results presentation (with or without AI-predicted stances labels) can affect the stance diversity consumption and search behavior of opinionated users on three debated topics (i.e., atheism, intellectual property rights, and school uniforms). Our results show that exposing participants to (counter-attitudinally) biased search results increases their consumption of attitude-opposing content, but we also found that bias was associated with a trend toward overall fewer interactions within the search page. We also found that 19% of users interacted with queries and search pages but did not select any search results. When we removed these participants in a post-hoc analysis, we found that stance labels increased the diversity of stances consumed by users, particularly when the search results were biased. Our findings highlight the need for future research to explore distinct search scenario settings to gain insight into opinionated users' behavior.