# Footprints of Data in a Classifier: Understanding the Privacy Risks and Solution Strategies

arXiv: https://arxiv.org/abs/2407.02268 · HTML: https://mlsanity.com/paper/2407.02268/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 123 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Adversarial vulnerability for any classifier](https://mlsanity.com/paper/1802.08686/) (2018)
- [Distributionally Robust Losses for Latent Covariate Mixtures](https://mlsanity.com/paper/2007.13982/) (2020)
- [PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models](https://mlsanity.com/paper/2204.01321/) (2022)
- [Machine Learning Security in Industry: A Quantitative Survey](https://mlsanity.com/paper/2207.05164/) (2022)
- [Model Extraction Attacks Revisited](https://mlsanity.com/paper/2312.05386/) (2023)
- [An Overview of Privacy-enhancing Technologies in Biometric Recognition](https://mlsanity.com/paper/2206.10465/) (2022)
- [Fast Exact Search in Hamming Space with Multi-Index Hashing](https://mlsanity.com/paper/1307.2982/) (2013)
- [A Nonparametric Ensemble Binary Classifier and its Statistical Properties](https://mlsanity.com/paper/1804.10928/) (2018)
- [Superensemble Classifier for Improving Predictions in Imbalanced Datasets](https://mlsanity.com/paper/1810.11317/) (2018)

## Built on by (most cited first, 1 of 1)

- [Implicit Identity Technologies for LLMs: Fingerprinting and Watermarking across Datasets, Models, and Generated Content](https://mlsanity.com/paper/2605.29245/) (2026)

## Abstract (arXiv)

The widespread deployment of Artificial Intelligence (AI) across government and private industries brings both advancements and heightened privacy and security concerns. Article 17 of the General Data Protection Regulation (GDPR) mandates the Right to Erasure, requiring data to be permanently removed from a system to prevent potential compromise. While existing research primarily focuses on erasing sensitive data attributes, several passive data compromise mechanisms remain underexplored and unaddressed. One such issue arises from the residual footprints of training data embedded within predictive models. Performance disparities between test and training data can inadvertently reveal which data points were part of the training set, posing a privacy risk. This study examines how two fundamental aspects of classifier systems - training data quality and classifier training methodology - contribute to privacy vulnerabilities. Our theoretical analysis demonstrates that classifiers exhibit universal vulnerability under conditions of data imbalance and distributional shifts. Empirical findings reinforce our theoretical results, highlighting the significant role of training data quality in classifier susceptibility. Additionally, our study reveals that a classifier's operational mechanism and architectural design impact its vulnerability. We further investigate mitigation strategies through data obfuscation techniques and analyze their impact on both privacy and classification performance. To aid practitioners, we introduce a privacy-performance trade-off index, providing a structured approach to balancing privacy protection with model effectiveness. The findings offer valuable insights for selecting classifiers and curating training data in diverse real-world applications.