# A Structured Analysis of Journalistic Evaluations for News Source Reliability

arXiv: https://arxiv.org/abs/2205.02736 · HTML: https://mlsanity.com/paper/2205.02736/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2022.

## Built on by (most cited first, 2 of 2)

- [Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models](https://mlsanity.com/paper/2410.18122/) (2024)
- [From Articles to Publishers: Aggregating Language Model Predictions for News Source Reliability Inference](https://mlsanity.com/paper/2609.24219/) (2026)

## Abstract (arXiv)

In today's era of information disorder, many organizations are moving to verify the veracity of news published on the web and social media. In particular, some agencies are exploring the world of online media and, through a largely manual process, ranking the credibility and transparency of news sources around the world. In this paper, we evaluate two procedures for assessing the risk of online media exposing their readers to m/disinformation. The procedures have been dictated by NewsGuard and The Global Disinformation Index, two well-known organizations combating d/misinformation via practices of good journalism. Specifically, considering a fixed set of media outlets, we examine how many of them were rated equally by the two procedures, and which aspects led to disagreement in the assessment. The result of our analysis shows a good degree of agreement, which in our opinion has a double value: it fortifies the correctness of the procedures and lays the groundwork for their automation.