Probabilistic Verification of Fairness Properties via Concentration
Not on the map. Why?
Abstract (arXiv)
As machine learning systems are increasingly used to make real world legal and financial decisions, it is of paramount importance that we develop algorithms to verify that these systems do not discriminate against minorities. We design a scalable algorithm for verifying fairness specifications. Our algorithm obtains strong correctness guarantees based on adaptive concentration inequalities; such inequalities enable our algorithm to adaptively take samples until it has enough data to make a decision. We implement our algorithm in a tool called VeriFair, and show that it scales to large machine learning models, including a deep recurrent neural network that is more than five orders of magnitude larger than the largest previously-verified neural network. While our technique only gives probabilistic guarantees due to the use of random samples, we show that we can choose the probability of error to be extremely small.
Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Dec 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 8; 2022: 12; 2023: 6; 2024: 6; 2025: 9; 2026: 4.
45 papers.
Sort:
By area:
- 2021Trustworthy AI: A Computational Perspectivecited by 93
- 2022Causality-based Neural Network Repaircited by 44
- 2021Fair Normalizing Flowscited by 38
- 2022Individual Fairness Guarantees for Neural Networkscited by 23
- 2022Fairify: Fairness Verification of Neural Networkscited by 22
- 2021Latent Space Smoothing for Individually Fair Representationscited by 18
- 2021Probabilistic Verification of Neural Networks Against Group Fairnesscited by 17
- 2022CertiFair: A Framework for Certified Global Fairness of Neural Networkscited by 17
- 2021Algorithmic Fairness Verification with Graphical Modelscited by 16
- 2021Automatic Fairness Testing of Neural Classifiers through Adversarial Samplingcited by 14
- 2024FairProof : Confidential and Certifiable Fairness for Neural Networkscited by 13
- 2022Certifying Some Distributional Fairness with Subpopulation Decompositioncited by 11
33 more
- 2024Certifying Counterfactual Bias in LLMscited by 9
- 2023Monitoring Algorithmic Fairnesscited by 8
- 2022Verifying Fairness in Quantum Machine Learningcited by 7
- 2023Certifying the Fairness of KNN in the Presence of Dataset Biascited by 7
- 2023Monitoring Algorithmic Fairness under Partial Observationscited by 7
- 2021Fairness Testing of Deep Image Classification with Adequacy Metricscited by 6
- 2022Provable Fairness for Neural Network Models using Formal Verificationcited by 6
- 2022Specifying and Testing k-Safety Properties for Machine-Learning Modelscited by 5
- 2022Explainable Global Fairness Verification of Tree-Based Classifierscited by 5
- 2023Adversarial Sampling for Fairness Testing in Deep Neural Networkcited by 5
- 2025Stream-Based Monitoring of Algorithmic Fairnesscited by 5
- 2022FETA: Fairness Enforced Verifying, Training, and Predicting Algorithms for Neural Networkscited by 4
- 2024Uncertainty Quantification for Neurosymbolic Programs via Compositional Conformal Predictioncited by 4
- 2024Solving Probabilistic Verification Problems of Neural Networks using Branch and Boundcited by 4
- 2024FairQuant: Certifying and Quantifying Fairness of Deep Neural Networkscited by 4
- 2022Certifying Fairness of Probabilistic Circuitscited by 3
- 2025FairSense: Long-Term Fairness Analysis of ML-Enabled Systemscited by 3
- 2023A Tale of Two Approximations: Tightening Over-Approximation for DNN Robustness Verification via Under-Approximationcited by 2
- 2025Correct-By-Construction: Certified Individual Fairness through Neural Network Trainingcited by 1
- 2026Analyzing Fairness of Neural Network Prediction via Counterfactual Dataset Generationcited by 1
- 2021Synthesizing Machine Learning Programs with PAC Guarantees via Statistical Sketchingcited by 0
- 2022DualApp: Tight Over-Approximation for Neural Network Robustness Verification via Under-Approximationcited by 0
- 2023Verification of Neural Networks Local Differential Classification Privacycited by 0
- 2024Probabilistic ML Verification via Weighted Model Integrationcited by 0
- 2025FairSHAP: Preprocessing for Fairness Through Attribution-Based Data Augmentationcited by 0
- 2025Language Models That Walk the Talk: A Framework for Formal Fairness Certificatescited by 0
- 2025Monitoring of Static Fairnesscited by 0
- 2025Algorithmic Fairness: A Runtime Perspectivecited by 0
- 2025The Fair Game: Auditing & Debiasing AI Algorithms Over Timecited by 0
- 2025Wasserstein projection distance for fairness testing of regression modelscited by 0
- 2026Shapley-Guided Neural Repair Approach via Derivative-Free Optimizationcited by 0
- 2026Energy Shields for Fairnesscited by 0
- 2026Beyond Binary: Continuous State Optimization with Graph-Structured Objectivescited by 0
- 2026Beyond Binary: Continuous State Optimization with Graph-Structured Objectives10 Aug 2026
- 2026Energy Shields for Fairness24 May 2026
- 2026Shapley-Guided Neural Repair Approach via Derivative-Free Optimization1 Apr 2026
- 2026Analyzing Fairness of Neural Network Prediction via Counterfactual Dataset Generation11 Feb 2026
- 2025Wasserstein projection distance for fairness testing of regression models5 Oct 2025
- 2025Correct-By-Construction: Certified Individual Fairness through Neural Network Training21 Aug 2025
- 2025The Fair Game: Auditing & Debiasing AI Algorithms Over Time8 Aug 2025
- 2025Algorithmic Fairness: A Runtime Perspective28 Jul 2025
- 2025Monitoring of Static Fairness3 Jul 2025
- 2025Language Models That Walk the Talk: A Framework for Formal Fairness Certificates19 May 2025
- 2025FairSHAP: Preprocessing for Fairness Through Attribution-Based Data Augmentation16 May 2025
- 2025Stream-Based Monitoring of Algorithmic Fairness30 Jan 2025
33 more
- 2025FairSense: Long-Term Fairness Analysis of ML-Enabled Systems3 Jan 2025
- 2024FairQuant: Certifying and Quantifying Fairness of Deep Neural Networks5 Sep 2024
- 2024Certifying Counterfactual Bias in LLMs29 May 2024
- 2024Solving Probabilistic Verification Problems of Neural Networks using Branch and Bound27 May 2024
- 2024Uncertainty Quantification for Neurosymbolic Programs via Compositional Conformal Prediction24 May 2024
- 2024FairProof : Confidential and Certifiable Fairness for Neural Networks19 Feb 2024
- 2024Probabilistic ML Verification via Weighted Model Integration7 Feb 2024
- 2023Verification of Neural Networks Local Differential Classification Privacy31 Oct 2023
- 2023Monitoring Algorithmic Fairness under Partial Observations1 Aug 2023
- 2023Certifying the Fairness of KNN in the Presence of Dataset Bias17 Jul 2023
- 2023A Tale of Two Approximations: Tightening Over-Approximation for DNN Robustness Verification via Under-Approximation26 May 2023
- 2023Monitoring Algorithmic Fairness25 May 2023
- 2023Adversarial Sampling for Fairness Testing in Deep Neural Network6 Mar 2023
- 2022Provable Fairness for Neural Network Models using Formal Verification16 Dec 2022
- 2022Fairify: Fairness Verification of Neural Networks8 Dec 2022
- 2022Certifying Fairness of Probabilistic Circuits5 Dec 2022
- 2022DualApp: Tight Over-Approximation for Neural Network Robustness Verification via Under-Approximation21 Nov 2022
- 2022Explainable Global Fairness Verification of Tree-Based Classifiers27 Sep 2022
- 2022Verifying Fairness in Quantum Machine Learning22 Jul 2022
- 2022Specifying and Testing k-Safety Properties for Machine-Learning Models13 Jun 2022
- 2022FETA: Fairness Enforced Verifying, Training, and Predicting Algorithms for Neural Networks1 Jun 2022
- 2022Certifying Some Distributional Fairness with Subpopulation Decomposition31 May 2022
- 2022CertiFair: A Framework for Certified Global Fairness of Neural Networks20 May 2022
- 2022Individual Fairness Guarantees for Neural Networks11 May 2022
- 2022Causality-based Neural Network Repair20 Apr 2022
- 2021Latent Space Smoothing for Individually Fair Representations26 Nov 2021
- 2021Fairness Testing of Deep Image Classification with Adequacy Metrics17 Nov 2021
- 2021Synthesizing Machine Learning Programs with PAC Guarantees via Statistical Sketching11 Oct 2021
- 2021Algorithmic Fairness Verification with Graphical Models20 Sep 2021
- 2021Probabilistic Verification of Neural Networks Against Group Fairness18 Jul 2021
- 2021Automatic Fairness Testing of Neural Classifiers through Adversarial Sampling17 Jul 2021
- 2021Trustworthy AI: A Computational Perspective12 Jul 2021
- 2021Fair Normalizing Flows10 Jun 2021
- 2022Causality-based Neural Network Repaircited by 44
- 2024Certifying Counterfactual Bias in LLMscited by 9
- 2026Shapley-Guided Neural Repair Approach via Derivative-Free Optimizationcited by 0
- 2026Shapley-Guided Neural Repair Approach via Derivative-Free Optimization1 Apr 2026
- 2024Certifying Counterfactual Bias in LLMs29 May 2024
- 2022Causality-based Neural Network Repair20 Apr 2022
- 2021Trustworthy AI: A Computational Perspectivecited by 93
- 2021Fair Normalizing Flowscited by 38
- 2022Individual Fairness Guarantees for Neural Networkscited by 23
- 2022Fairify: Fairness Verification of Neural Networkscited by 22
- 2021Latent Space Smoothing for Individually Fair Representationscited by 18
- 2021Probabilistic Verification of Neural Networks Against Group Fairnesscited by 17
- 2022CertiFair: A Framework for Certified Global Fairness of Neural Networkscited by 17
- 2021Algorithmic Fairness Verification with Graphical Modelscited by 16
- 2021Automatic Fairness Testing of Neural Classifiers through Adversarial Samplingcited by 14
- 2024FairProof : Confidential and Certifiable Fairness for Neural Networkscited by 13
- 2022Certifying Some Distributional Fairness with Subpopulation Decompositioncited by 11
- 2023Monitoring Algorithmic Fairnesscited by 8
12 most cited of 42 All 42 (JSON)
- 2026Beyond Binary: Continuous State Optimization with Graph-Structured Objectives10 Aug 2026
- 2026Energy Shields for Fairness24 May 2026
- 2026Analyzing Fairness of Neural Network Prediction via Counterfactual Dataset Generation11 Feb 2026
- 2025Wasserstein projection distance for fairness testing of regression models5 Oct 2025
- 2025Correct-By-Construction: Certified Individual Fairness through Neural Network Training21 Aug 2025
- 2025The Fair Game: Auditing & Debiasing AI Algorithms Over Time8 Aug 2025
- 2025Algorithmic Fairness: A Runtime Perspective28 Jul 2025
- 2025Monitoring of Static Fairness3 Jul 2025
- 2025Language Models That Walk the Talk: A Framework for Formal Fairness Certificates19 May 2025
- 2025FairSHAP: Preprocessing for Fairness Through Attribution-Based Data Augmentation16 May 2025
- 2025Stream-Based Monitoring of Algorithmic Fairness30 Jan 2025
- 2025FairSense: Long-Term Fairness Analysis of ML-Enabled Systems3 Jan 2025
12 newest of 42 All 42 (JSON)