mlsanityData to 7 Oct 2026

FairSense: Long-Term Fairness Analysis of ML-Enabled Systems

arXiv 2501.01665 ↗First posted 3 Jan 2025cs.LG cs.CY cs.SEcited by 3

Not on the map. Why?

Abstract (arXiv)

Algorithmic fairness of machine learning (ML) models has raised significant concern in the recent years. Many testing, verification, and bias mitigation techniques have been proposed to identify and reduce fairness issues in ML models. The existing methods are model-centric and designed to detect fairness issues under static settings. However, many ML-enabled systems operate in a dynamic environment where the predictive decisions made by the system impact the environment, which in turn affects future decision-making. Such a self-reinforcing feedback loop can cause fairness violations in the long term, even if the immediate outcomes are fair. In this paper, we propose a simulation-based framework called FairSense to detect and analyze long-term unfairness in ML-enabled systems. Given a fairness requirement, FairSense performs Monte-Carlo simulation to enumerate evolution traces for each system configuration. Then, FairSense performs sensitivity analysis on the space of possible configurations to understand the impact of design options and environmental factors on the long-term fairness of the system. We demonstrate FairSense's potential utility through three real-world case studies: Loan lending, opioids risk scoring, and predictive policing.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 45 of its 102 references that are in the map. First posted on arXiv 3 Jan 2025.

Builds on

45 of 102 references.

References · 45

  1. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2014Certifying and removing disparate impactcited by 715
  4. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  5. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  6. 2018A comparative study of fairness-enhancing interventions in machine learningcited by 208
  7. 2018Delayed Impact of Fair Machine Learningcited by 183
  8. 2017Fairness Testing: Testing Software for Discriminationcited by 123
  9. 2017Runaway Feedback Loops in Predictive Policingcited by 97
  10. 2021Bias in Machine Learning Software: Why? How? What to do?cited by 92
  11. 2018Automated Directed Fairness Testingcited by 77
  12. 2022Data Feedback Loops: Model-driven Amplification of Dataset Biasescited by 59
33 more
  1. 2019Algorithms for Fairness in Sequential Decision Makingcited by 53
  2. 2020Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairnesscited by 52
  3. 2020Verifying Individual Fairness in Machine Learning Modelscited by 49
  4. 2021Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipelinecited by 49
  5. 2018Probabilistic Verification of Fairness Properties via Concentrationcited by 45
  6. 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 42
  7. 2021NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identificationcited by 42
  8. 2018From Fair Decision Making to Social Equalitycited by 36
  9. 2022Fairness-aware Configuration of Machine Learning Librariescited by 28
  10. 2023Long-Term Fairness with Unknown Dynamicscited by 25
  11. 2022Marrying Fairness and Explainability in Supervised Learningcited by 23
  12. 2022Fairify: Fairness Verification of Neural Networkscited by 22
  13. 2020Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Contextcited by 18
  14. 2022Achieving Long-Term Fairness in Sequential Decision Makingcited by 18
  15. 2022Adaptive Fairness Improvement Based on Causality Analysiscited by 17
  16. 2023Runtime Monitoring of Dynamic Fairness Propertiescited by 17
  17. 2021The effect of differential victim crime reporting on predictive policing systemscited by 15
  18. 2020VerifyMed -- A blockchain platform for transparent trust in virtualized healthcare: Proof-of-conceptcited by 14
  19. 2021Fair Enough: Searching for Sufficient Measures of Fairnesscited by 14
  20. 2022The Long Arc of Fairness: Formalisations and Ethical Discoursecited by 12
  21. 2023A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systemscited by 12
  22. 2020Astraea: Grammar-based Fairness Testingcited by 11
  23. 2023Information-Theoretic Testing and Debugging of Fairness Defects in Deep Neural Networkscited by 10
  24. 2023Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoMLcited by 9
  25. 2023Monitoring Algorithmic Fairnesscited by 8
  26. 2022How Biased are Your Features?: Computing Fairness Influence Functions with Global Sensitivity Analysiscited by 7
  27. 2022Enforcing Delayed-Impact Fairness Guaranteescited by 7
  28. 2023Certifying the Fairness of KNN in the Presence of Dataset Biascited by 7
  29. 2024Long-Term Fairness Inquiries and Pursuits in Machine Learning: A Survey of Notions, Methods, and Challengescited by 4
  30. 2020Black Loans Matter: Distributionally Robust Fairness for Fighting Subgroup Discriminationcited by 3
  31. 2021On Adaptive Fairness in Software Systemscited by 2
  32. 2022System Safety Engineering for Social and Ethical ML Risks: A Case Studycited by 2
  33. 2024Long-term Fairness For Real-time Decision Making: A Constrained Online Optimization Approachcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Rethinking Autonomy: Preventing Failures in AI-Driven Software Engineeringcited by 4
  2. 2025RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contextscited by 3
  3. 2026Auditing Fairness under Model Updates: Fundamental Complexity and Property-Preserving Updatescited by 1