# Enabling SQL-based Training Data Debugging for Federated Learning

arXiv: https://arxiv.org/abs/2108.11884 · HTML: https://mlsanity.com/paper/2108.11884/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 65 references that are in the map.
- First posted on arXiv 26 Aug 2021.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Privacy Preserving Vertical Federated Learning for Tree-based Models](https://mlsanity.com/paper/2008.06170/) (2020)
- [HoloClean: Holistic Data Repairs with Probabilistic Inference](https://mlsanity.com/paper/1702.00820/) (2017)
- [Practical Federated Gradient Boosting Decision Trees](https://mlsanity.com/paper/1911.04206/) (2019)
- [Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator](https://mlsanity.com/paper/1911.09824/) (2019)
- [Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions](https://mlsanity.com/paper/1901.10501/) (2019)
- [FOCUS: Dealing with Label Quality Disparity in Federated Learning](https://mlsanity.com/paper/2001.11359/) (2020)
- [Towards Federated Graph Learning for Collaborative Financial Crimes Detection](https://mlsanity.com/paper/1909.12946/) (2019)
- [HoloDetect: Few-Shot Learning for Error Detection](https://mlsanity.com/paper/1904.02285/) (2019)
- [A Data Quality-Driven View of MLOps](https://mlsanity.com/paper/2102.07750/) (2021)
- [Automated Machine Learning: State-of-The-Art and Open Challenges](https://mlsanity.com/paper/1906.02287/) (2019)
- [FederBoost: Private Federated Learning for GBDT](https://mlsanity.com/paper/2011.02796/) (2020)
- [BoostClean: Automated Error Detection and Repair for Machine Learning](https://mlsanity.com/paper/1711.01299/) (2017)
- [Model Assertions for Monitoring and Improving ML Models](https://mlsanity.com/paper/2003.01668/) (2020)
- [Complaint-driven Training Data Debugging for Query 2.0](https://mlsanity.com/paper/2004.05722/) (2020)
- [Quality Inference in Federated Learning with Secure Aggregation](https://mlsanity.com/paper/2007.06236/) (2020)
- [Privacy-preserving Decentralized Aggregation for Federated Learning](https://mlsanity.com/paper/2012.07183/) (2020)

## Built on by (most cited first, 8 of 8)

- [A Survey for Federated Learning Evaluations: Goals and Measures](https://mlsanity.com/paper/2308.11841/) (2023)
- [Eliminating Label Leakage in Tree-Based Vertical Federated Learning](https://mlsanity.com/paper/2307.10318/) (2023)
- [Secure and Verifiable Data Collaboration with Low-Cost Zero-Knowledge Proofs](https://mlsanity.com/paper/2311.15310/) (2023)
- [FedST: Secure Federated Shapelet Transformation for Time Series Classification](https://mlsanity.com/paper/2302.10631/) (2023)
- [Data Debugging is NP-hard for Classifiers Trained with SGD](https://mlsanity.com/paper/2408.01365/) (2024)
- [Comparative Evaluation of Data Decoupling Techniques for Federated Machine Learning with Database as a Service](https://mlsanity.com/paper/2303.08371/) (2023)
- [Enhancing Data Provenance and Model Transparency in Federated Learning Systems -- A Database Approach](https://mlsanity.com/paper/2403.01451/) (2024)
- [ProToken: Token-Level Attribution for Federated Large Language Models](https://mlsanity.com/paper/2601.19672/) (2026)

## Abstract (arXiv)

How can we debug a logistical regression model in a federated learning setting when seeing the model behave unexpectedly (e.g., the model rejects all high-income customers' loan applications)? The SQL-based training data debugging framework has proved effective to fix this kind of issue in a non-federated learning setting. Given an unexpected query result over model predictions, this framework automatically removes the label errors from training data such that the unexpected behavior disappears in the retrained model. In this paper, we enable this powerful framework for federated learning. The key challenge is how to develop a security protocol for federated debugging which is proved to be secure, efficient, and accurate. Achieving this goal requires us to investigate how to seamlessly integrate the techniques from multiple fields (Databases, Machine Learning, and Cybersecurity). We first propose FedRain, which extends Rain, the state-of-the-art SQL-based training data debugging framework, to our federated learning setting. We address several technical challenges to make FedRain work and analyze its security guarantee and time complexity. The analysis results show that FedRain falls short in terms of both efficiency and security. To overcome these limitations, we redesign our security protocol and propose Frog, a novel SQL-based training data debugging framework tailored for federated learning. Our theoretical analysis shows that Frog is more secure, more accurate, and more efficient than FedRain. We conduct extensive experiments using several real-world datasets and a case study. The experimental results are consistent with our theoretical analysis and validate the effectiveness of Frog in practice.