# Should we trust web-scraped data?

arXiv: https://arxiv.org/abs/2308.02231 · HTML: https://mlsanity.com/paper/2308.02231/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 62 references that are in the map.
- First posted on arXiv 4 Aug 2023.

## Builds on (most cited first, 2 of 2)

- [TED-On: A Total Error Framework for Digital Traces of Human Behavior on Online Platforms](https://mlsanity.com/paper/1907.08228/) (2019)
- [Incorrect Data in the Widely Used Inside Airbnb Dataset](https://mlsanity.com/paper/2007.03019/) (2020)

## Built on by (most cited first, 2 of 2)

- [STICKERCONV: Generating Multimodal Empathetic Responses from Scratch](https://mlsanity.com/paper/2402.01679/) (2024)
- [Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems](https://mlsanity.com/paper/2410.23472/) (2024)

## Abstract (arXiv)

The increasing adoption of econometric and machine-learning approaches by empirical researchers has led to a widespread use of one data collection method: web scraping. Web scraping refers to the use of automated computer programs to access websites and download their content. The key argument of this paper is that na"ive web scraping procedures can lead to sampling bias in the collected data. This article describes three sources of sampling bias in web-scraped data. More specifically, sampling bias emerges from web content being volatile (i.e., being subject to change), personalized (i.e., presented in response to request characteristics), and unindexed (i.e., abundance of a population register). In a series of examples, I illustrate the prevalence and magnitude of sampling bias. To support researchers and reviewers, this paper provides recommendations on anticipating, detecting, and overcoming sampling bias in web-scraped data.