# Auditing YouTube's Recommendation Algorithm for Misinformation Filter Bubbles

arXiv: https://arxiv.org/abs/2210.10085 · HTML: https://mlsanity.com/paper/2210.10085/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 60 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [Degenerate Feedback Loops in Recommender Systems](https://mlsanity.com/paper/1902.10730/) (2019)
- [Auditing Radicalization Pathways on YouTube](https://mlsanity.com/paper/1908.08313/) (2019)
- [Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors](https://mlsanity.com/paper/2105.02980/) (2021)
- [Towards Automatic Detection of Misinformation in Online Medical Videos](https://mlsanity.com/paper/1909.01543/) (2019)
- [Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits](https://mlsanity.com/paper/2102.04256/) (2021)
- [Measuring Personalization of Web Search](https://mlsanity.com/paper/1706.05011/) (2017)
- [An Audit of Misinformation Filter Bubbles on YouTube: Bubble Bursting and Recent Behavior Changes](https://mlsanity.com/paper/2203.13769/) (2022)
- [Auditing E-Commerce Platforms for Algorithmically Curated Vaccine Misinformation](https://mlsanity.com/paper/2101.08419/) (2021)
- ["It is just a flu": Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations](https://mlsanity.com/paper/2010.11638/) (2020)
- [YouTube, The Great Radicalizer? Auditing and Mitigating Ideological Biases in YouTube Recommendations](https://mlsanity.com/paper/2203.10666/) (2022)
- [Personality Traits and Echo Chambers on Facebook](https://mlsanity.com/paper/1606.04721/) (2016)
- [Facebook Ads Monitor: An Independent Auditing System for Political Ads on Facebook](https://mlsanity.com/paper/2001.10581/) (2020)
- [Emotional Dynamics in the Age of Misinformation](https://mlsanity.com/paper/1505.08001/) (2015)
- [Advanced Machine Learning Techniques for Fake News (Online Disinformation) Detection: A Systematic Mapping Study](https://mlsanity.com/paper/2101.01142/) (2020)
- [Analysing the Effect of Recommendation Algorithms on the Amplification of Misinformation](https://mlsanity.com/paper/2103.14748/) (2021)
- [Conspiracy Brokers: Understanding the Monetization of YouTube Conspiracy Theories](https://mlsanity.com/paper/2205.15943/) (2022)
- [A Study of Fake News Reading and Annotating in Social Media Context](https://mlsanity.com/paper/2109.12523/) (2021)

## Built on by (most cited first, 8 of 8)

- [Human-AI Coevolution](https://mlsanity.com/paper/2306.13723/) (2023)
- [Harmful YouTube Video Detection: A Taxonomy of Online Harm and MLLMs as Alternative Annotators](https://mlsanity.com/paper/2411.05854/) (2024)
- [Prompt-to-Slate: Diffusion Models for Prompt-Conditioned Slate Generation](https://mlsanity.com/paper/2408.06883/) (2024)
- [Identifying Misinformation on YouTube through Transcript Contextual Analysis with Transformer Models](https://mlsanity.com/paper/2307.12155/) (2023)
- [Reducing Exposure to Harmful Content via Graph Rewiring](https://mlsanity.com/paper/2306.07930/) (2023)
- [Iffy-Or-Not: Extending the Web to Support the Critical Evaluation of Fallacious Texts](https://mlsanity.com/paper/2503.14412/) (2025)
- [The DSA's Blind Spot: Algorithmic Audit of Advertising and Minor Profiling on TikTok](https://mlsanity.com/paper/2603.05653/) (2026)
- [Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale](https://mlsanity.com/paper/2606.30801/) (2026)

## Abstract (arXiv)

In this paper, we present results of an auditing study performed over YouTube aimed at investigating how fast a user can get into a misinformation filter bubble, but also what it takes to "burst the bubble", i.e., revert the bubble enclosure. We employ a sock puppet audit methodology, in which pre-programmed agents (acting as YouTube users) delve into misinformation filter bubbles by watching misinformation promoting content. Then they try to burst the bubbles and reach more balanced recommendations by watching misinformation debunking content. We record search results, home page results, and recommendations for the watched videos. Overall, we recorded 17,405 unique videos, out of which we manually annotated 2,914 for the presence of misinformation. The labeled data was used to train a machine learning model classifying videos into three classes (promoting, debunking, neutral) with the accuracy of 0.82. We use the trained model to classify the remaining videos that would not be feasible to annotate manually. Using both the manually and automatically annotated data, we observe the misinformation bubble dynamics for a range of audited topics. Our key finding is that even though filter bubbles do not appear in some situations, when they do, it is possible to burst them by watching misinformation debunking content (albeit it manifests differently from topic to topic). We also observe a sudden decrease of misinformation filter bubble effect when misinformation debunking videos are watched after misinformation promoting videos, suggesting a strong contextuality of recommendations. Finally, when comparing our results with a previous similar study, we do not observe significant improvements in the overall quantity of recommended misinformation content.