mlsanityData to 14 Oct 2026

Privacy Drift: Evolving Privacy Concerns in Incremental Learning

arXiv 2412.05183 ↗First posted 6 Dec 2024cs.LG cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

In the evolving landscape of machine learning (ML), Federated Learning (FL) presents a paradigm shift towards decentralized model training while preserving user data privacy. This paper introduces the concept of ``privacy drift", an innovative framework that parallels the well-known phenomenon of concept drift. While concept drift addresses the variability in model accuracy over time due to changes in the data, privacy drift encapsulates the variation in the leakage of private information as models undergo incremental training. By defining and examining privacy drift, this study aims to unveil the nuanced relationship between the evolution of model performance and the integrity of data privacy. Through rigorous experimentation, we investigate the dynamics of privacy drift in FL systems, focusing on how model updates and data distribution shifts influence the susceptibility of models to privacy attacks, such as membership inference attacks (MIA). Our results highlight a complex interplay between model accuracy and privacy safeguards, revealing that enhancements in model performance can lead to increased privacy risks. We provide empirical evidence from experiments on customized datasets derived from CIFAR-100 (Canadian Institute for Advanced Research, 100 classes), showcasing the impact of data and concept drift on privacy. This work lays the groundwork for future research on privacy-aware machine learning, aiming to achieve a delicate balance between model accuracy and data privacy in decentralized environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 22 references that are in the map. First posted on arXiv 6 Dec 2024.

Builds on

14 of 22 references.

References · 14

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2019Large Scale Incremental Learningcited by 732
  6. 2020Learning under Concept Drift: A Reviewcited by 477
  7. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  8. 2022NVIDIA FLARE: Federated Learning from Simulation to Real-Worldcited by 80
  9. 2022Adapting to Online Label Shift with Provable Guaranteescited by 26
  10. 2021Automatically detecting data drift in machine learning classifierscited by 15
  11. 2021Accuracy-Privacy Trade-off in Deep Ensemble: A Membership Inference Perspectivecited by 10
  12. 2023MIA-BAD: An Approach for Enhancing Membership Inference Attack and its Mitigation with Federated Learningcited by 7
2 more
  1. 2020Concept Drift and Covariate Shift Detection Ensemble with Lagged Labelscited by 5
  2. 2024Accuracy-Privacy Trade-off in the Mitigation of Membership Inference Attack in Federated Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Privacy in Federated Learning
  2. 2023Privacy and Accuracy Implications of Model Complexity and Integration in Heterogeneous Federated Learning
  3. 2025Evaluating the Dynamics of Membership Privacy in Deep Learning
  4. 2024Analysis of Privacy Leakage in Federated Large Language Models
  5. 2023Privacy Threats in Stable Diffusion Models
  6. 2023Exploring Federated Unlearning: Review, Comparison, and Insights
  7. 2026Evaluating Differential Privacy Against Membership Inference in Federated Learning: Insights from the NIST Genomics Red Team Challenge
  8. 2026Local Layer-wise Differential Privacy in Federated Learning