Software Doping Analysis for Human Oversight
Not on the map. Why?
Abstract (arXiv)
This article introduces a framework that is meant to assist in mitigating societal risks that software can pose. Concretely, this encompasses facets of software doping as well as unfairness and discrimination in high-risk decision-making systems. The term software doping refers to software that contains surreptitiously added functionality that is against the interest of the user. A prominent example of software doping are the tampered emission cleaning systems that were found in millions of cars around the world when the diesel emissions scandal surfaced. The first part of this article combines the formal foundations of software doping analysis with established probabilistic falsification techniques to arrive at a black-box analysis technique for identifying undesired effects of software. We apply this technique to emission cleaning systems in diesel cars but also to high-risk systems that evaluate humans in a possibly unfair or discriminating way. We demonstrate how our approach can assist humans-in-the-loop to make better informed and more responsible decisions. This is to promote effective human oversight, which will be a central requirement enforced by the European Union's upcoming AI Act. We complement our technical contribution with a juridically, philosophically, and psychologically informed perspective on the potential problems caused by such systems.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 152 references that are in the map. First posted on arXiv 11 Aug 2023.
Builds on
26 of 152 references.
References · 26
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
- 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
- 2016Model-Agnostic Interpretability of Machine Learningcited by 219
- 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 166
- 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
- 2019On the Apparent Conflict Between Individual and Group Fairnesscited by 126
- 2020Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challengescited by 120
- 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
- 2018iFair: Learning Individually Fair Data Representations for Algorithmic Decision Makingcited by 64
- 2020Two Simple Ways to Learn Individual Fairness Metrics from Datacited by 54
14 more
- 2019Metric Learning for Individual Fairnesscited by 51
- 2020Verifying Individual Fairness in Machine Learning Modelscited by 49
- 2020A systematic review and taxonomy of explanations in decision support and recommender systemscited by 45
- 2021Fairness in Ranking: A Surveycited by 33
- 2020Bias and Discrimination in AI: a cross-disciplinary perspectivecited by 24
- 2014Temporal Logics for Hyperpropertiescited by 19
- 2021On the Relation of Trust and Explainability: Why to Engineer for Trustworthinesscited by 16
- 2021Exploring Explainability: A Definition, a Model, and a Knowledge Cataloguecited by 15
- 2021Explainability Auditing for Intelligent Systems: A Rationale for Multi-Disciplinary Perspectivescited by 7
- 2019Doping Tests for Cyber-Physical Systemscited by 2
- 2017Is your software on dope? Formal analysis of surreptitiously "enhanced" programscited by 1
- 2018Facets of Software Dopingcited by 1
- 2020Conformance Relations and Hyperproperties for Doping Detection in Time and Spacecited by 1
- 2021RTLola on Board: Testing Real Driving Emissions on your Phonecited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.