# Accurate Measures of Vaccination and Concerns of Vaccine Holdouts from Web Search Logs

arXiv: https://arxiv.org/abs/2306.07457 · HTML: https://mlsanity.com/paper/2306.07457/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 96 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Learning from positive and unlabeled data: a survey](https://mlsanity.com/paper/1811.04820/) (2018)
- [COVID-19 Vaccine Hesitancy on Social Media: Building a Public Twitter Dataset of Anti-vaccine Content, Vaccine Misinformation and Conspiracies](https://mlsanity.com/paper/2105.05134/) (2021)
- [Winds of Change: Impact of COVID-19 on Vaccine-related Opinions of Twitter users](https://mlsanity.com/paper/2111.10667/) (2021)
- [Online misinformation is linked to early COVID-19 vaccination hesitancy and refusal](https://mlsanity.com/paper/2104.10635/) (2021)
- [Detecting disparities in police deployments using dashcam data](https://mlsanity.com/paper/2305.15210/) (2023)
- [Using Search Queries to Understand Health Information Needs in Africa](https://mlsanity.com/paper/1806.05740/) (2018)
- [Google COVID-19 Vaccination Search Insights: Anonymization Process Description](https://mlsanity.com/paper/2107.01179/) (2021)
- [From Cookies to Cooks: Insights on Dietary Patterns via Analysis of Web Usage Logs](https://mlsanity.com/paper/1304.3742/) (2013)
- [Population-Scale Study of Human Needs During the COVID-19 Pandemic: Analysis and Implications](https://mlsanity.com/paper/2008.07045/) (2020)
- [Vaccine Search Patterns Provide Insights into Vaccination Intent](https://mlsanity.com/paper/2111.11424/) (2021)

## Built on by (most cited first, 3 of 3)

- [Can A Society of Generative Agents Simulate Human Behavior and Inform Public Health Policy? A Case Study on Vaccine Hesitancy](https://mlsanity.com/paper/2503.09639/) (2025)
- [Learning production functions for supply chains with graph neural networks](https://mlsanity.com/paper/2407.18772/) (2024)
- [Urban Incident Prediction with Graph Neural Networks: Integrating Government Ratings and Crowdsourced Reports](https://mlsanity.com/paper/2506.08740/) (2025)

## Abstract (arXiv)

To design effective vaccine policies, policymakers need detailed data about who has been vaccinated, who is holding out, and why. However, existing data in the US are insufficient: reported vaccination rates are often delayed or missing, and surveys of vaccine hesitancy are limited by high-level questions and self-report biases. Here, we show how large-scale search engine logs and machine learning can be leveraged to fill these gaps and provide novel insights about vaccine intentions and behaviors. First, we develop a vaccine intent classifier that can accurately detect when a user is seeking the COVID-19 vaccine on search. Our classifier demonstrates strong agreement with CDC vaccination rates, with correlations above 0.86, and estimates vaccine intent rates to the level of ZIP codes in real time, allowing us to pinpoint more granular trends in vaccine seeking across regions, demographics, and time. To investigate vaccine hesitancy, we use our classifier to identify two groups, vaccine early adopters and vaccine holdouts. We find that holdouts, compared to early adopters matched on covariates, are 69% more likely to click on untrusted news sites. Furthermore, we organize 25,000 vaccine-related URLs into a hierarchical ontology of vaccine concerns, and we find that holdouts are far more concerned about vaccine requirements, vaccine development and approval, and vaccine myths, and even within holdouts, concerns vary significantly across demographic groups. Finally, we explore the temporal dynamics of vaccine concerns and vaccine seeking, and find that key indicators emerge when individuals convert from holding out to preparing to accept the vaccine.