mlsanityData to 14 Oct 2026

Monitoring of Static Fairness

arXiv 2507.03048 ↗First posted 3 Jul 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Machine-learned systems are in widespread use for making decisions about humans, and it is important that they are fair, i.e., not biased against individuals based on sensitive attributes. We present a general framework of runtime verification of algorithmic fairness for systems whose models are unknown, but are assumed to have a Markov chain structure, with or without full observation of the state space. We introduce a specification language that can model many common algorithmic fairness properties, such as demographic parity, equal opportunity, and social burden. We build monitors that observe a long sequence of events as generated by a given system, and output, after each observation, a quantitative estimate of how fair or biased the system was on that run until that point in time. The estimate is proven to be correct modulo a variable error bound and a given confidence level, where the error bound gets tighter as the observed sequence gets longer. We present two categories of monitoring algorithms, namely ones with a uniform error bound across all time points, and ones with weaker non-uniform, pointwise error bounds at different time points. Our monitoring algorithms use statistical tools that are adapted to suit the dynamic requirements of monitoring and the special needs of the fairness specifications. Using a prototype implementation, we show how we can monitor if a bank is fair in giving loans to applicants from different social backgrounds, and if a college is fair in admitting students while maintaining a reasonable financial burden on the society. In these experiments, our monitors took less than a millisecond to update their verdicts after each observation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 70 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

34 of 70 references.

References · 34

  1. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  2. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  3. 2014Certifying and removing disparate impactcited by 715
  4. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 672
  5. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 586
  6. 2017Algorithmic decision making and the cost of fairnesscited by 403
  7. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 384
  8. 2018Time-uniform, nonparametric, nonasymptotic confidence sequencescited by 229
  9. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  10. 2018Delayed Impact of Fair Machine Learningcited by 184
  11. 2020CheXclusion: Fairness gaps in deep chest X-ray classifierscited by 153
  12. 2017A Convex Framework for Fair Regressioncited by 148
22 more
  1. 2018The Social Cost of Strategic Classificationcited by 105
  2. 2017Runaway Feedback Loops in Predictive Policingcited by 97
  3. 2018Differentially Private Fair Learningcited by 76
  4. 2018iFair: Learning Individually Fair Data Representations for Algorithmic Decision Makingcited by 64
  5. 2020Verifying Individual Fairness in Machine Learning Modelscited by 49
  6. 2019Average Individual Fairness: Algorithms, Generalization and Experimentscited by 46
  7. 2018Probabilistic Verification of Fairness Properties via Concentrationcited by 45
  8. 2021Fair Normalizing Flowscited by 38
  9. 2019PAC Statistical Model Checking for Markov Decision Processes and Stochastic Gamescited by 32
  10. 2020Justicia: A Stochastic SAT Approach to Formally Verify Fairnesscited by 25
  11. 2021Certifying Robustness to Programmable Data Bias in Decision Treescited by 21
  12. 2021Fairness-Aware PAC Learning from Corrupted Datacited by 18
  13. 2021Probabilistic Verification of Neural Networks Against Group Fairnesscited by 17
  14. 2023Runtime Monitoring of Dynamic Fairness Propertiescited by 17
  15. 2021Algorithmic Fairness Verification with Graphical Modelscited by 16
  16. 2017Real-time Stream-based Monitoringcited by 13
  17. 2023Monitoring Algorithmic Fairnesscited by 8
  18. 2023Monitoring Algorithmic Fairness under Partial Observationscited by 7
  19. 2019Monitoring Event Frequenciescited by 5
  20. 2024Fairness Shields: Safeguarding against Biased Decision Makerscited by 5
  21. 2021Quantitative and Approximate Monitoringcited by 3
  22. 2025Monitoring Robustness and Individual Fairnesscited by 3

Built on by

Not cited yet.