# Black-Box Audits for Group Distribution Shifts

arXiv: https://arxiv.org/abs/2209.03620 · HTML: https://mlsanity.com/paper/2209.03620/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 8 Sep 2022.

## Builds on (most cited first, 9 of 9)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifiers](https://mlsanity.com/paper/1306.4447/) (2013)
- [Dataset Inference: Ownership Resolution in Machine Learning](https://mlsanity.com/paper/2104.10706/) (2021)
- [Formalizing and Estimating Distribution Inference Risks](https://mlsanity.com/paper/2109.06024/) (2021)

## Built on by (most cited first, 5 of 5)

- [Dissecting Distribution Inference](https://mlsanity.com/paper/2212.07591/) (2022)
- [Manipulating Transfer Learning for Property Inference](https://mlsanity.com/paper/2303.11643/) (2023)
- [Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs](https://mlsanity.com/paper/2601.03087/) (2026)
- [Attesting Distributional Properties of Training Data for Machine Learning](https://mlsanity.com/paper/2308.09552/) (2023)
- [Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks](https://mlsanity.com/paper/2507.20708/) (2025)

## Abstract (arXiv)

When a model informs decisions about people, distribution shifts can create undue disparities. However, it is hard for external entities to check for distribution shift, as the model and its training set are often proprietary. In this paper, we introduce and study a black-box auditing method to detect cases of distribution shift that lead to a performance disparity of the model across demographic groups. By extending techniques used in membership and property inference attacks -- which are designed to expose private information from learned models -- we demonstrate that an external auditor can gain the information needed to identify these distribution shifts solely by querying the model. Our experimental results on real-world datasets show that this approach is effective, achieving 80--100% AUC-ROC in detecting shifts involving the underrepresentation of a demographic group in the training set. Researchers and investigative journalists can use our tools to perform non-collaborative audits of proprietary models and expose cases of underrepresentation in the training datasets.