# Reliability Estimation of News Media Sources: Birds of a Feather Flock Together

arXiv: https://arxiv.org/abs/2404.09565 · HTML: https://mlsanity.com/paper/2404.09565/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 20 of 20)

- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning](https://mlsanity.com/paper/1809.06416/) (2018)
- [The science of fake news](https://mlsanity.com/paper/2307.07903/) (2023)
- [That is a Known Lie: Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2005.06058/) (2020)
- [Predicting Factuality of Reporting and Bias of News Media Sources](https://mlsanity.com/paper/1810.01765/) (2018)
- [What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context](https://mlsanity.com/paper/2005.04518/) (2020)
- [A Survey on Truth Discovery](https://mlsanity.com/paper/1505.02463/) (2015)
- [Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media](https://mlsanity.com/paper/1904.00542/) (2019)
- [NELA-GT-2018: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/1904.01546/) (2019)
- [Knowledge-Based Trust: Estimating the Trustworthiness of Web Sources](https://mlsanity.com/paper/1502.03519/) (2015)
- [BIC: Twitter Bot Detection with Text-Graph Interaction and Semantic Consistency](https://mlsanity.com/paper/2208.08320/) (2022)
- [NELA-GT-2019: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/2003.08444/) (2020)
- [Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction](https://mlsanity.com/paper/2207.14116/) (2022)
- [People on Media: Jointly Identifying Credible News and Trustworthy Citizen Journalists in Online Communities](https://mlsanity.com/paper/1705.02667/) (2017)
- [Exposing Paid Opinion Manipulation Trolls](https://mlsanity.com/paper/2109.13726/) (2021)
- [Reinforcement Ranking](https://mlsanity.com/paper/1303.5988/) (2013)
- [IDIAPers @ Causal News Corpus 2022: Efficient Causal Relation Identification Through a Prompt-based Few-shot Approach](https://mlsanity.com/paper/2209.03895/) (2022)

## Built on by (most cited first, 7 of 7)

- [Retrieval-Augmented Generation with Estimation of Source Reliability](https://mlsanity.com/paper/2410.22954/) (2024)
- [Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models](https://mlsanity.com/paper/2410.18122/) (2024)
- [Mapping the Media Landscape: Predicting Factual Reporting and Political Bias Through Web Interactions](https://mlsanity.com/paper/2410.17655/) (2024)
- [Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact](https://mlsanity.com/paper/2605.14021/) (2026)
- [Analyzing the temporal dynamics of linguistic features contained in misinformation](https://mlsanity.com/paper/2503.04786/) (2025)
- [A Multi-View Media Profiling Suite: Resources, Evaluation, and Analysis](https://mlsanity.com/paper/2605.01336/) (2026)
- [From Articles to Publishers: Aggregating Language Model Predictions for News Source Reliability Inference](https://mlsanity.com/paper/2609.24219/) (2026)

## Abstract (arXiv)

Evaluating the reliability of news sources is a routine task for journalists and organizations committed to acquiring and disseminating accurate information. Recent research has shown that predicting sources' reliability represents an important first-prior step in addressing additional challenges such as fake news detection and fact-checking. In this paper, we introduce a novel approach for source reliability estimation that leverages reinforcement learning strategies for estimating the reliability degree of news sources. Contrary to previous research, our proposed approach models the problem as the estimation of a reliability degree, and not a reliability label, based on how all the news media sources interact with each other on the Web. We validated the effectiveness of our method on a news media reliability dataset that is an order of magnitude larger than comparable existing datasets. Results show that the estimated reliability degrees strongly correlates with journalists-provided scores (Spearman=0.80) and can effectively predict reliability labels (macro-avg. F1 score=81.05). We release our implementation and dataset, aiming to provide a valuable resource for the NLP community working on information verification.