# Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomes

arXiv: https://arxiv.org/abs/2307.05862 · HTML: https://mlsanity.com/paper/2307.05862/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 12 Jul 2023.

## Builds on (most cited first, 17 of 17)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [An Investigation of Why Overparameterization Exacerbates Spurious Correlations](https://mlsanity.com/paper/2005.04345/) (2020)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution](https://mlsanity.com/paper/1608.01041/) (2016)
- [Disparities in Dermatology AI: Assessments Using Diverse Clinical Images](https://mlsanity.com/paper/2111.08006/) (2021)
- [Algorithmic Monoculture and Social Welfare](https://mlsanity.com/paper/2101.05853/) (2021)
- [Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?](https://mlsanity.com/paper/2211.13972/) (2022)
- [From Facial Expression Recognition to Interpersonal Relation Prediction](https://mlsanity.com/paper/1609.06426/) (2016)
- [Ecosystem Graphs: The Social Footprint of Foundation Models](https://mlsanity.com/paper/2303.15772/) (2023)
- [Manipulation-Proof Machine Learning](https://mlsanity.com/paper/2004.03865/) (2020)
- [Did the Model Change? Efficiently Assessing Machine Learning API Shifts](https://mlsanity.com/paper/2107.14203/) (2021)
- [HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions](https://mlsanity.com/paper/2209.08443/) (2022)
- [Should attention be all we need? The epistemic and ethical implications of unification in machine learning](https://mlsanity.com/paper/2205.08377/) (2022)
- [The Right to be an Exception to a Data-Driven Rule](https://mlsanity.com/paper/2212.13995/) (2022)

## Built on by (most cited first, 10 of 10)

- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Correlated Errors in Large Language Models](https://mlsanity.com/paper/2506.07962/) (2025)
- [Human Expertise in Algorithmic Prediction](https://mlsanity.com/paper/2402.00793/) (2024)
- [CIRCLE: A Framework for Evaluating AI from a Real-World Lens](https://mlsanity.com/paper/2602.24055/) (2026)
- [The Subjectivity of Monoculture](https://mlsanity.com/paper/2602.24086/) (2026)
- [Algorithmic Monocultures in Hiring](https://mlsanity.com/paper/2605.27371/) (2026)
- [Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift](https://mlsanity.com/paper/2606.25152/) (2026)
- [The Hitchhiker's Guide to Monoculture: AI Homogenizes Syntax, Not (Necessarily) Semantics](https://mlsanity.com/paper/2607.13077/) (2026)
- [Why Better Models Can Create Riskier Systems: Evidence from LLM Agents in Financial Markets](https://mlsanity.com/paper/2609.04373/) (2026)
- [Who Audits Whom, on What Substrate, with What Evidence? An Independence-Graded Audit Protocol for Agentic AI](https://mlsanity.com/paper/2609.18272/) (2026)

## Abstract (arXiv)

Machine learning is traditionally studied at the model level: researchers measure and improve the accuracy, robustness, bias, efficiency, and other dimensions of specific models. In practice, the societal impact of machine learning is determined by the surrounding context of machine learning deployments. To capture this, we introduce ecosystem-level analysis: rather than analyzing a single model, we consider the collection of models that are deployed in a given context. For example, ecosystem-level analysis in hiring recognizes that a job candidate's outcomes are not only determined by a single hiring algorithm or firm but instead by the collective decisions of all the firms they applied to. Across three modalities (text, images, speech) and 11 datasets, we establish a clear trend: deployed machine learning is prone to systemic failure, meaning some users are exclusively misclassified by all models available. Even when individual models improve at the population level over time, we find these improvements rarely reduce the prevalence of systemic failure. Instead, the benefits of these improvements predominantly accrue to individuals who are already correctly classified by other models. In light of these trends, we consider medical imaging for dermatology where the costs of systemic failure are especially high. While traditional analyses reveal racial performance disparities for both models and humans, ecosystem-level analysis reveals new forms of racial disparity in model predictions that do not present in human predictions. These examples demonstrate ecosystem-level analysis has unique strengths for characterizing the societal impact of machine learning.