# Audits Under Resource, Data, and Access Constraints: Scaling Laws For Less Discriminatory Alternatives

arXiv: https://arxiv.org/abs/2509.05627 · HTML: https://mlsanity.com/paper/2509.05627/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 65 references that are in the map.
- First posted on arXiv 6 Sep 2025.

## Builds on (most cited first, 29 of 29)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Learning the Pareto Front with Hypernetworks](https://mlsanity.com/paper/2010.04104/) (2020)
- [Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance](https://mlsanity.com/paper/2206.04737/) (2022)
- [The Gradient of Generative AI Release: Methods and Considerations](https://mlsanity.com/paper/2302.04844/) (2023)
- [Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem](https://mlsanity.com/paper/2310.02521/) (2023)
- [A Safe Harbor for AI Evaluation and Red Teaming](https://mlsanity.com/paper/2403.04893/) (2024)
- [Scalable Pareto Front Approximation for Deep Multi-Objective Learning](https://mlsanity.com/paper/2103.13392/) (2021)
- [Accuracy and Fairness Trade-offs in Machine Learning: A Stochastic Multi-Objective Approach](https://mlsanity.com/paper/2008.01132/) (2020)
- [Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling](https://mlsanity.com/paper/2402.17861/) (2024)
- [A survey on measuring indirect discrimination in machine learning](https://mlsanity.com/paper/1511.00148/) (2015)
- [Trustless Audits without Revealing Data or Models](https://mlsanity.com/paper/2404.04500/) (2024)
- [From Transparency to Accountability and Back: A Discussion of Access and Evidence in AI Auditing](https://mlsanity.com/paper/2410.04772/) (2024)
- [Sociotechnical Audits: Broadening the Algorithm Auditing Lens to Investigate Targeted Advertising](https://mlsanity.com/paper/2308.15768/) (2023)
- [Bayes-Optimal Fair Classification with Linear Disparity Constraints via Pre-, In-, and Post-processing](https://mlsanity.com/paper/2402.02817/) (2024)
- [Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing](https://mlsanity.com/paper/2104.01634/) (2021)
- [The Legal Duty to Search for Less Discriminatory Algorithms](https://mlsanity.com/paper/2406.06817/) (2024)
- [Law and the Emerging Political Economy of Algorithmic Audits](https://mlsanity.com/paper/2406.11855/) (2024)
- [A Hybrid 2-stage Neural Optimization for Pareto Front Extraction](https://mlsanity.com/paper/2101.11684/) (2021)
- [Consider the Alternatives: Navigating Fairness-Accuracy Tradeoffs via Disqualification](https://mlsanity.com/paper/2110.00813/) (2021)

## Built on by (most cited first, 2 of 2)

- [Fairness vs Performance: Characterizing the Pareto Frontier of Algorithmic Decision Systems](https://mlsanity.com/paper/2605.10604/) (2026)
- [Statistical Guarantees in the Search for Less Discriminatory Algorithms](https://mlsanity.com/paper/2512.23943/) (2025)

## Abstract (arXiv)

AI audits play a critical role in AI accountability and safety. One branch of the law for which AI audits are particularly salient is anti-discrimination law. Several areas of anti-discrimination law implicate the "less discriminatory alternative" (LDA) requirement, in which a protocol (e.g., model) is defensible if no less discriminatory protocol that achieves comparable performance can be found with a reasonable amount of effort. Notably, the burden of proving an LDA exists typically falls on the claimant (the party alleging discrimination). This creates a significant hurdle in AI cases, as the claimant would seemingly need to train a less discriminatory yet high-performing model, a task requiring resources and expertise beyond most litigants. Moreover, developers often shield information about and access to their model and training data as trade secrets, making it difficult to reproduce a similar model from scratch. In this work, we present a procedure enabling claimants to determine if an LDA exists, even when they have limited compute, data, information, and model access. We focus on the setting in which fairness is given by demographic parity and performance by binary cross-entropy loss. As our main result, we provide a novel closed-form upper bound for the loss-fairness Pareto frontier (PF). We show how the claimant can use it to fit a PF in the "low-resource regime," then extrapolate the PF that applies to the (large) model being contested, all without training a single large model. The expression thus serves as a scaling law for loss-fairness PFs. To use this scaling law, the claimant would require a small subsample of the train/test data. Then, the claimant can fit the context-specific PF by training as few as 7 (small) models. We stress test our main result in simulations, finding that our scaling law holds even when the exact conditions of our theory do not.