# Introducing explainable supervised machine learning into interactive feedback loops for statistical production system

arXiv: https://arxiv.org/abs/2202.03212 · HTML: https://mlsanity.com/paper/2202.03212/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Algorithmic Recourse: from Counterfactual Explanations to Interventions](https://mlsanity.com/paper/2002.06278/) (2020)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [Leveraging Unlabeled Data to Predict Out-of-Distribution Performance](https://mlsanity.com/paper/2201.04234/) (2022)
- [Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks](https://mlsanity.com/paper/2107.07455/) (2021)
- [Explainable AI for Trees: From Local Explanations to Global Understanding](https://mlsanity.com/paper/1905.04610/) (2019)
- [Continual Learning in Practice](https://mlsanity.com/paper/1903.05202/) (2019)
- [Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features](https://mlsanity.com/paper/2104.00629/) (2021)
- [Monitoring Model Deterioration with Explainable Uncertainty Estimation via Non-parametric Bootstrap](https://mlsanity.com/paper/2201.11676/) (2022)
- [Fairness Implications of Encoding Protected Categorical Attributes](https://mlsanity.com/paper/2201.11358/) (2022)
- [Copying Machine Learning Classifiers](https://mlsanity.com/paper/1903.01879/) (2019)
- [Quantile Encoder: Tackling High Cardinality Categorical Features in Regression Problems](https://mlsanity.com/paper/2105.13783/) (2021)
- [Desiderata for Explainable AI in statistical production systems of the European Central Bank](https://mlsanity.com/paper/2107.08045/) (2021)
- [Sampling Unknown Decision Functions to Build Classifier Copies](https://mlsanity.com/paper/1910.00237/) (2019)
- [Towards Global Explanations for Credit Risk Scoring](https://mlsanity.com/paper/1811.07698/) (2018)

## Abstract (arXiv)

Statistical production systems cover multiple steps from the collection, aggregation, and integration of data to tasks like data quality assurance and dissemination. While the context of data quality assurance is one of the most promising fields for applying machine learning, the lack of curated and labeled training data is often a limiting factor. The statistical production system for the Centralised Securities Database features an interactive feedback loop between data collected by the European Central Bank and data quality assurance performed by data quality managers at National Central Banks. The quality assurance feedback loop is based on a set of rule-based checks for raising exceptions, upon which the user either confirms the data or corrects an actual error. In this paper we use the information received from this feedback loop to optimize the exceptions presented to the National Central Banks thereby improving the quality of exceptions generated and the time consumed on the system by the users authenticating those exceptions. For this approach we make use of explainable supervised machine learning to (a) identify the types of exceptions and (b) to prioritize which exceptions are more likely to require an intervention or correction by the NCBs. Furthermore, we provide an explainable AI taxonomy aiming to identify the different explainable AI needs that arose during the project.