# Do You Think It's Biased? How To Ask For The Perception Of Media Bias

arXiv: https://arxiv.org/abs/2112.07392 · HTML: https://mlsanity.com/paper/2112.07392/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 41 references that are in the map.
- First posted on arXiv 14 Dec 2021.

## Builds on (most cited first, 4 of 4)

- [In Plain Sight: Media Bias Through the Lens of Factual Reporting](https://mlsanity.com/paper/1909.02670/) (2019)
- [Analyzing Political Bias and Unfairness in News Articles at Different Levels of Granularity](https://mlsanity.com/paper/2010.10652/) (2020)
- [MBIC -- A Media Bias Annotation Dataset Including Annotator Characteristics](https://mlsanity.com/paper/2105.11910/) (2021)
- [Enabling News Consumers to View and Understand Biased News Coverage: A Study on the Perception and Visualization of Media Bias](https://mlsanity.com/paper/2105.09640/) (2021)

## Built on by (most cited first, 12 of 12)

- [Neural Media Bias Detection Using Distant Supervision With BABE -- Bias Annotations By Experts](https://mlsanity.com/paper/2209.14557/) (2022)
- [NeuS: Neutral Multi-News Summarization for Mitigating Framing Bias](https://mlsanity.com/paper/2204.04902/) (2022)
- [A Domain-adaptive Pre-training Approach for Language Bias Detection in News](https://mlsanity.com/paper/2205.10773/) (2022)
- [The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection](https://mlsanity.com/paper/2411.11081/) (2024)
- [The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias](https://mlsanity.com/paper/2312.16148/) (2023)
- [Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles](https://mlsanity.com/paper/2211.03491/) (2022)
- [An Interdisciplinary Approach for the Automated Detection and Visualization of Media Bias in News Articles](https://mlsanity.com/paper/2112.13352/) (2021)
- [MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressions](https://mlsanity.com/paper/2403.07910/) (2024)
- [Bias or Diversity? Unraveling Fine-Grained Thematic Discrepancy in U.S. News Headlines](https://mlsanity.com/paper/2303.15708/) (2023)
- [Mitigating Framing Bias with Polarity Minimization Loss](https://mlsanity.com/paper/2311.01817/) (2023)
- [Through the LLM Looking Glass: A Socratic Probing of Donkeys, Elephants, and Markets](https://mlsanity.com/paper/2503.16674/) (2025)
- [Definitional Sensitivity in Media Bias Detection: A Multi-Definition Dataset and Benchmark](https://mlsanity.com/paper/2608.23095/) (2026)

## Abstract (arXiv)

Media coverage possesses a substantial effect on the public perception of events. The way media frames events can significantly alter the beliefs and perceptions of our society. Nevertheless, nearly all media outlets are known to report news in a biased way. While such bias can be introduced by altering the word choice or omitting information, the perception of bias also varies largely depending on a reader's personal background. Therefore, media bias is a very complex construct to identify and analyze. Even though media bias has been the subject of many studies, previous assessment strategies are oversimplified, lack overlap and empirical evaluation. Thus, this study aims to develop a scale that can be used as a reliable standard to evaluate article bias. To name an example: Intending to measure bias in a news article, should we ask, "How biased is the article?" or should we instead ask, "How did the article treat the American president?". We conducted a literature search to find 824 relevant questions about text perception in previous research on the topic. In a multi-iterative process, we summarized and condensed these questions semantically to conclude a complete and representative set of possible question types about bias. The final set consisted of 25 questions with varying answering formats, 17 questions using semantic differentials, and six ratings of feelings. We tested each of the questions on 190 articles with overall 663 participants to identify how well the questions measure an article's perceived bias. Our results show that 21 final items are suitable and reliable for measuring the perception of media bias. We publish the final set of questions on http://bias-question-tree.gipplab.org/.