# Media Cloud: Massive Open Source Collection of Global News on the Open Web

arXiv: https://arxiv.org/abs/2104.03702 · HTML: https://mlsanity.com/paper/2104.03702/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2021.

## Built on by (most cited first, 20 of 20)

- [MN-DS: A Multilabeled News Dataset for News Articles Hierarchical Classification](https://mlsanity.com/paper/2212.12061/) (2022)
- [NELA-GT-2022: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/2203.05659/) (2022)
- [LLM Teacher-Student Framework for Text Classification With No Manually Annotated Data: A Case Study in IPTC News Topic Classification](https://mlsanity.com/paper/2411.19638/) (2024)
- [DiscoverLLM: From Executing Intents to Discovering Them](https://mlsanity.com/paper/2602.03429/) (2026)
- [Identifying and Investigating Global News Coverage of Critical Events Such as Disasters and Terrorist Attacks](https://mlsanity.com/paper/2506.12925/) (2025)
- [It's the Same Old Story! Enriching Event-Centric Knowledge Graphs by Narrative Aspects](https://mlsanity.com/paper/2205.03876/) (2022)
- [Mapping News Narratives Using LLMs and Narrative-Structured Text Embeddings](https://mlsanity.com/paper/2409.06540/) (2024)
- [From Nuisance to News Sense: Augmenting the News with Cross-Document Evidence and Context](https://mlsanity.com/paper/2310.04592/) (2023)
- [Lost in Recursion: Mining Rich Event Semantics in Knowledge Graphs](https://mlsanity.com/paper/2404.16405/) (2024)
- [NewsHomepages: Homepage Layouts Capture Information Prioritization Decisions](https://mlsanity.com/paper/2501.00004/) (2024)
- [Beyond the Battlefield: Framing Analysis of Media Coverage in Conflict Reporting](https://mlsanity.com/paper/2506.10421/) (2025)
- [Infini-News: Efficiently Queryable Access to 1.3 Billion Processed Common Crawl News Articles](https://mlsanity.com/paper/2605.18337/) (2026)
- [Artificial Intelligence in Brazilian News: A Mixed-Methods Analysis](https://mlsanity.com/paper/2410.17423/) (2024)
- [Author Unknown: Evaluating Performance of Author Extraction Libraries on Global Online News Articles](https://mlsanity.com/paper/2410.19771/) (2024)
- [Auditing Google's Search Algorithm: Measuring News Diversity Across Brazil, the UK, and the US](https://mlsanity.com/paper/2410.23842/) (2024)
- [Multilingualism, Transnationality, and K-pop in the Online #StopAsianHate Movement](https://mlsanity.com/paper/2503.02707/) (2025)
- [BAR-Analytics: A Web-based Platform for Analyzing Information Spreading Barriers in News: Comparative Analysis Across Multiple Barriers and Events](https://mlsanity.com/paper/2503.24220/) (2025)
- [SemCAFE: When Named Entities make the Difference Assessing Web Source Reliability through Entity-level Analytics](https://mlsanity.com/paper/2504.08776/) (2025)
- [FRaN-X: FRaming and Narratives-eXplorer](https://mlsanity.com/paper/2507.06974/) (2025)
- [From Web(logs) to Web(AI): Questions, Platforms, and Methods across Twenty Editions of ICWSM](https://mlsanity.com/paper/2610.00205/) (2026)

## Abstract (arXiv)

We present the first full description of Media Cloud, an open source platform based on crawling hyperlink structure in operation for over 10 years, that for many uses will be the best way to collect data for studying the media ecosystem on the open web. We document the key choices behind what data Media Cloud collects and stores, how it processes and organizes these data, and its open API access as well as user-facing tools. We also highlight the strengths and limitations of the Media Cloud collection strategy compared to relevant alternatives. We give an overview two sample datasets generated using Media Cloud and discuss how researchers can use the platform to create their own datasets.