mlsanityData to 14 Oct 2026

Private Learning with Public Features

arXiv 2310.15454 ↗First posted 24 Oct 2023cs.LG cs.CR stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

We study a class of private learning problems in which the data is a join of private and public features. This is often the case in private personalization tasks such as recommendation or ad prediction, in which features related to individuals are sensitive, while features related to items (the movies or songs to be recommended, or the ads to be shown to users) are publicly available and do not require protection. A natural question is whether private algorithms can achieve higher utility in the presence of public features. We give a positive answer for multi-encoder models where one of the encoders operates on public features. We develop new algorithms that take advantage of this separation by only protecting certain sufficient statistics (instead of adding noise to the gradient). This method has a guaranteed utility improvement for linear regression, and importantly, achieves the state of the art on two standard private recommendation benchmarks, demonstrating the importance of methods that adapt to the private-public feature separation.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 31 of its 65 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

31 of 65 references.

References · 31

  1. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2019Deep Leakage from Gradientscited by 1,021
  4. 2016Wide & Deep Learning for Recommender Systemscited by 747
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  6. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  7. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  8. 2018Variational Autoencoders for Collaborative Filteringcited by 310
  9. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 224
  10. 2021Deep Learning with Label Differential Privacycited by 117
  11. 2020Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identificationcited by 90
  12. 2021Large Scale Private Learning via Low-rank Reparametrizationcited by 88
19 more
  1. 2021Federated Reconstruction: Partially Local Federated Learningcited by 79
  2. 2018Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domaincited by 61
  3. 2022Mixed Differential Privacy in Computer Visioncited by 54
  4. 2021Public Data-Assisted Mirror Descent for Private Model Trainingcited by 49
  5. 2019On the Difficulty of Evaluating Baselines: A Study on Recommender Systemscited by 46
  6. 2015Efficient Per-Example Gradient Computationscited by 42
  7. 2022Private Adaptive Optimization with Side Informationcited by 38
  8. 2023Why Is Public Pretraining Necessary for Private Model Training?cited by 37
  9. 2020Long-tail Session-based Recommendationcited by 33
  10. 2020Learning from Mixtures of Private and Public Populationscited by 19
  11. 2021Wide Network Learning with Differential Privacycited by 17
  12. 2022Regression with Label Differential Privacycited by 15
  13. 2023Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learningcited by 13
  14. 2021Private Alternating Least Squares: Practical Private Matrix Completion with Tighter Ratescited by 10
  15. 2022Differentially Private Vertical Federated Clusteringcited by 10
  16. 2017Differentially Private Matrix Completion Revisitedcited by 8
  17. 2022Algorithms with More Granular Differential Privacy Guaranteescited by 4
  18. 2023Multi-Task Differential Privacy Under Distribution Skewcited by 3
  19. 2023Private Matrix Factorization with Public Item Featurescited by 3

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 1.

6 papers.

  1. 2024Training Differentially Private Ad Prediction Models with Semi-Sensitive Featurescited by 6
  2. 2024On the Linear Speedup of Personalized Federated Reinforcement Learning with Shared Representationscited by 4
  3. 2024Locally Private Estimation with Public Featurescited by 3
  4. 2024Enhancing Learning with Label Differential Privacy by Vector Approximationcited by 2
  5. 2025Machine Learning with Privacy for Protected Attributescited by 2
  6. 2024On Convex Optimization with Semi-Sensitive Featurescited by 1