# Neural Media Bias Detection Using Distant Supervision With BABE -- Bias Annotations By Experts

arXiv: https://arxiv.org/abs/2209.14557 · HTML: https://mlsanity.com/paper/2209.14557/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 29 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [In Plain Sight: Media Bias Through the Lens of Factual Reporting](https://mlsanity.com/paper/1909.02670/) (2019)
- [Analyzing Political Bias and Unfairness in News Articles at Different Levels of Granularity](https://mlsanity.com/paper/2010.10652/) (2020)
- [MBIC -- A Media Bias Annotation Dataset Including Annotator Characteristics](https://mlsanity.com/paper/2105.11910/) (2021)
- [Neural Based Statement Classification for Biased Language](https://mlsanity.com/paper/1811.05740/) (2018)
- [Enabling News Consumers to View and Understand Biased News Coverage: A Study on the Perception and Visualization of Media Bias](https://mlsanity.com/paper/2105.09640/) (2021)
- [Do You Think It's Biased? How To Ask For The Perception Of Media Bias](https://mlsanity.com/paper/2112.07392/) (2021)
- [Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification](https://mlsanity.com/paper/1703.02504/) (2017)

## Built on by (most cited first, 30 of 56)

- [Measuring Political Bias in Large Language Models: What Is Said and How It Is Said](https://mlsanity.com/paper/2403.18932/) (2024)
- [NBIAS: A Natural Language Processing Framework for Bias Identification in Text](https://mlsanity.com/paper/2308.01681/) (2023)
- [Introducing MBIB -- the first Media Bias Identification Benchmark Task and Dataset Collection](https://mlsanity.com/paper/2304.13148/) (2023)
- [IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators](https://mlsanity.com/paper/2402.00345/) (2024)
- [The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection](https://mlsanity.com/paper/2411.11081/) (2024)
- [The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias](https://mlsanity.com/paper/2312.16148/) (2023)
- [Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles](https://mlsanity.com/paper/2211.03491/) (2022)
- [Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary](https://mlsanity.com/paper/2411.10461/) (2024)
- [Quantifying Generative Media Bias with a Corpus of Real-world and Generated News Articles](https://mlsanity.com/paper/2406.10773/) (2024)
- [Predicting Sentence-Level Factuality of News and Bias of Media Outlets](https://mlsanity.com/paper/2301.11850/) (2023)
- [Disentangling Structure and Style: Political Bias Detection in News by Inducing Document Hierarchy](https://mlsanity.com/paper/2304.02247/) (2023)
- [Connecting the Dots in News Analysis: Bridging the Cross-Disciplinary Disparities in Media Bias and Framing](https://mlsanity.com/paper/2309.08069/) (2023)
- [ICLEF: In-Context Learning with Expert Feedback for Explainable Style Transfer](https://mlsanity.com/paper/2309.08583/) (2023)
- [To Bias or Not to Bias: Detecting bias in News with bias-detector](https://mlsanity.com/paper/2505.13010/) (2025)
- [PRISM: A Framework for Producing Interpretable Political Bias Embeddings with Political-Aware Cross-Encoder](https://mlsanity.com/paper/2505.24646/) (2025)
- [Unlocking Bias Detection: Leveraging Transformer-Based Models for Content Analysis](https://mlsanity.com/paper/2310.00347/) (2023)
- [MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressions](https://mlsanity.com/paper/2403.07910/) (2024)
- [Analyzing Multi-Task Learning for Abstractive Text Summarization](https://mlsanity.com/paper/2210.14606/) (2022)
- [Sentence-level Media Bias Analysis with Event Relation Graph](https://mlsanity.com/paper/2404.01722/) (2024)
- [Pitfalls of Conversational LLMs on News Debiasing](https://mlsanity.com/paper/2404.06488/) (2024)
- [CS-Insights: A System for Analyzing Computer Science Research](https://mlsanity.com/paper/2210.06878/) (2022)
- [Data-Augmented and Retrieval-Augmented Context Enrichment in Chinese Media Bias Detection](https://mlsanity.com/paper/2311.01372/) (2023)
- [MGM: Global Understanding of Audience Overlap Graphs for Predicting the Factuality and the Bias of News Media](https://mlsanity.com/paper/2412.10467/) (2024)
- [Leveraging Large Language Models for Automated Definition Extraction with TaxoMatic A Case Study on Media Bias](https://mlsanity.com/paper/2504.00343/) (2025)
- [Multi-document Summarization through Multi-document Event Relation Graph Reasoning in LLMs: a case study in Framing Bias Mitigation](https://mlsanity.com/paper/2506.12978/) (2025)
- [Generate, Evaluate, Iterate: Synthetic Data for Human-in-the-Loop Refinement of LLM Judges](https://mlsanity.com/paper/2511.04478/) (2025)
- [Bike Frames: Understanding the Implicit Portrayal of Cyclists in the News](https://mlsanity.com/paper/2301.06178/) (2023)
- [Crossing the Aisle: Unveiling Partisan and Counter-Partisan Events in News Reporting](https://mlsanity.com/paper/2310.18768/) (2023)
- [Navigating News Narratives: A Media Bias Analysis Dataset](https://mlsanity.com/paper/2312.00168/) (2023)
- [On Context-aware Detection of Cherry-picking in News Reporting](https://mlsanity.com/paper/2401.05650/) (2024)

## Abstract (arXiv)

Media coverage has a substantial effect on the public perception of events. Nevertheless, media outlets are often biased. One way to bias news articles is by altering the word choice. The automatic identification of bias by word choice is challenging, primarily due to the lack of a gold standard data set and high context dependencies. This paper presents BABE, a robust and diverse data set created by trained experts, for media bias research. We also analyze why expert labeling is essential within this domain. Our data set offers better annotation quality and higher inter-annotator agreement than existing work. It consists of 3,700 sentences balanced among topics and outlets, containing media bias labels on the word and sentence level. Based on our data, we also introduce a way to detect bias-inducing sentences in news articles automatically. Our best performing BERT-based model is pre-trained on a larger corpus consisting of distant labels. Fine-tuning and evaluating the model on our proposed supervised data set, we achieve a macro F1-score of 0.804, outperforming existing methods.