# Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

arXiv: https://arxiv.org/abs/2102.04256 · HTML: https://mlsanity.com/paper/2102.04256/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 172 references that are in the map.
- First posted on arXiv 3 Feb 2021.

## Builds on (most cited first, 25 of 25)

- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Does Object Recognition Work for Everyone?](https://mlsanity.com/paper/1906.02659/) (2019)
- [Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search](https://mlsanity.com/paper/1905.01989/) (2019)
- [Towards a Critical Race Methodology in Algorithmic Fairness](https://mlsanity.com/paper/1912.03593/) (2019)
- [Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence](https://mlsanity.com/paper/2007.04068/) (2020)
- [Discrimination in Online Ad Delivery](https://mlsanity.com/paper/1301.6822/) (2013)
- [Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination](https://mlsanity.com/paper/1408.6491/) (2014)
- [Auditing Radicalization Pathways on YouTube](https://mlsanity.com/paper/1908.08313/) (2019)
- [Roles for Computing in Social Change](https://mlsanity.com/paper/1912.04883/) (2019)
- [Like trainer, like bot? Inheritance of bias in algorithmic content moderation](https://mlsanity.com/paper/1707.01477/) (2017)
- [Measuring Personalization of Web Search](https://mlsanity.com/paper/1706.05011/) (2017)
- [Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media](https://mlsanity.com/paper/1704.01347/) (2017)
- [A Mulching Proposal](https://mlsanity.com/paper/1908.06166/) (2019)
- [Facebook Ads Monitor: An Independent Auditing System for Political Ads on Facebook](https://mlsanity.com/paper/2001.10581/) (2020)
- [A Longitudinal Analysis of YouTube's Promotion of Conspiracy Videos](https://mlsanity.com/paper/2003.03318/) (2020)
- [Crowd-assisted Search for Price Discrimination in E-Commerce: First results](https://mlsanity.com/paper/1307.4531/) (2013)
- [Measuring the Importance of User-Generated Content to Search Engines](https://mlsanity.com/paper/1906.08576/) (2019)
- [Auditing News Curation Systems: A Case Study Examining Algorithmic and Editorial Logic in Apple News](https://mlsanity.com/paper/1908.00456/) (2019)
- [Ad Delivery Algorithms: The Hidden Arbiters of Political Messaging](https://mlsanity.com/paper/1912.04255/) (2019)
- [Gender Differences in Participation and Reward on Stack Overflow](https://mlsanity.com/paper/1811.11539/) (2018)
- [The Accuracy of the Demographic Inferences Shown on Google's Ad Settings](https://mlsanity.com/paper/1808.07549/) (2018)
- [You can't see what you can't see: Experimental evidence for how much relevant information may be missed due to Google's Web search personalisation](https://mlsanity.com/paper/1904.13022/) (2019)

## Built on by (most cited first, 21 of 21)

- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Evaluating and Mitigating Discrimination in Language Model Decisions](https://mlsanity.com/paper/2312.03689/) (2023)
- [A Survey on Bias in Visual Datasets](https://mlsanity.com/paper/2107.07919/) (2021)
- [Supporting Human-AI Collaboration in Auditing LLMs with LLMs](https://mlsanity.com/paper/2304.09991/) (2023)
- [A Comprehensive Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2209.10117/) (2022)
- [From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing](https://mlsanity.com/paper/2410.04772/) (2024)
- [System Cards for AI-Based Decision-Making for Public Policy](https://mlsanity.com/paper/2203.04754/) (2022)
- [Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations](https://mlsanity.com/paper/2503.01623/) (2025)
- [An External Stability Audit Framework to Test the Validity of Personality Prediction in AI Hiring](https://mlsanity.com/paper/2201.09151/) (2022)
- [Auditing of AI: Legal, Ethical and Technical Approaches](https://mlsanity.com/paper/2407.06235/) (2024)
- [Auditing YouTube's Recommendation Algorithm for Misinformation Filter Bubbles](https://mlsanity.com/paper/2210.10085/) (2022)
- [Addressing the regulatory gap: moving towards an EU AI audit ecosystem beyond the AI Act by including civil society](https://mlsanity.com/paper/2403.07904/) (2024)
- [Evidence of behavior consistent with self-interest and altruism in an artificially intelligent agent](https://mlsanity.com/paper/2301.02330/) (2023)
- [Measuring an artificial intelligence agent's trust in humans using machine incentives](https://mlsanity.com/paper/2212.13371/) (2022)
- [Certification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study](https://mlsanity.com/paper/2305.18307/) (2023)
- [Under manipulations, are some AI models harder to audit?](https://mlsanity.com/paper/2402.09043/) (2024)
- [Law and the Emerging Political Economy of Algorithmic Audits](https://mlsanity.com/paper/2406.11855/) (2024)
- [Reducing Exposure to Harmful Content via Graph Rewiring](https://mlsanity.com/paper/2306.07930/) (2023)
- [The Cadaver in the Machine: The Social Practices of Measurement and Validation in Motion Capture Technology](https://mlsanity.com/paper/2401.10877/) (2024)
- [Advancing AI Audits for Enhanced AI Governance](https://mlsanity.com/paper/2312.00044/) (2023)
- [Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communities](https://mlsanity.com/paper/2606.13397/) (2026)

## Abstract (arXiv)

While algorithm audits are growing rapidly in commonality and public importance, relatively little scholarly work has gone toward synthesizing prior work and strategizing future research in the area. This systematic literature review aims to do just that, following PRISMA guidelines in a review of over 500 English articles that yielded 62 algorithm audit studies. The studies are synthesized and organized primarily by behavior (discrimination, distortion, exploitation, and misjudgement), with codes also provided for domain (e.g. search, vision, advertising, etc.), organization (e.g. Google, Facebook, Amazon, etc.), and audit method (e.g. sock puppet, direct scrape, crowdsourcing, etc.). The review shows how previous audit studies have exposed public-facing algorithms exhibiting problematic behavior, such as search algorithms culpable of distortion and advertising algorithms culpable of discrimination. Based on the studies reviewed, it also suggests some behaviors (e.g. discrimination on the basis of intersectional identities), domains (e.g. advertising algorithms), methods (e.g. code auditing), and organizations (e.g. Twitter, TikTok, LinkedIn) that call for future audit attention. The paper concludes by offering the common ingredients of successful audits, and discussing algorithm auditing in the context of broader research working toward algorithmic justice.