# Fairness under Prediction Error in Predict-then-Optimize Pipelines

arXiv: https://arxiv.org/abs/2610.18593 · HTML: https://mlsanity.com/paper/2610.18593/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 8 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 6 of 6)

- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization](https://mlsanity.com/paper/1809.05504/) (2018)
- [Decision-centric fairness: Evaluation and optimization for resource allocation problems](https://mlsanity.com/paper/2504.20642/) (2025)
- [End-to-End Fairness Optimization with Fair Decision-Focused Learning](https://mlsanity.com/paper/2607.29441/) (2026)

## Abstract (arXiv)

Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes haty, since true outcomes y are unobserved at decision time. We study whether a demographic parity constraint enforced this way continues to hold against y, and how the resulting violation scales with prediction error. Using a budget-constrained allocation problem solved as a linear program, we show empirically that the true-outcome fairness gap grows with prediction error σ and saturates, well fit by c(1 - e-σ/τ) with c = 0.1545 ± 0.0103 and τ = 3.9045 ± 0.3745; its mean is invariant to sample size while its variance decays at the standard O(1/√n) rate. We derive a two-part theoretical bound: a closed-form local sensitivity result showing the gap's first-order response to prediction error depends on only two individuals per group, validated against the solver to within three significant figures; and a universal, error-independent ceiling following from the boundedness of the decision variable, confirmed consistent with but looser than the empirical saturation level. On US Census income data, enforcing demographic parity on a trained predictor's output reduces an unconstrained disparity in true income to roughly one-third of its original size, leaving 34.4% ± 3.3% (California) and 35.7% ± 1.8% (Texas) of the disparity intact and undetectable from the predictions alone -- consistent not only in mechanism but in approximate magnitude across two states with different demographics and income distributions, and stable across ten independent train/test splits in each. A fairness constraint that appears fully satisfied at decision time can mask a substantial, systematic violation of the outcome it was meant to protect.