Differentially Private Relational Learning with Entity-level Privacy Guarantees
Not on the map. Why?
Abstract (arXiv)
Learning with relational and network-structured data is increasingly vital in sensitive domains where protecting the privacy of individual entities is paramount. Differential Privacy (DP) offers a principled approach for quantifying privacy risks, with DP-SGD emerging as a standard mechanism for private model training. However, directly applying DP-SGD to relational learning is challenging due to two key factors: (i) entities often participate in multiple relations, resulting in high and difficult-to-control sensitivity; and (ii) relational learning typically involves multi-stage, potentially coupled (interdependent) sampling procedures that make standard privacy amplification analyses inapplicable. This work presents a principled framework for relational learning with formal entity-level DP guarantees. We provide a rigorous sensitivity analysis and introduce an adaptive gradient clipping scheme that modulates clipping thresholds based on entity occurrence frequency. We also extend the privacy amplification results to a tractable subclass of coupled sampling, where the dependence arises only through sample sizes. These contributions lead to a tailored DP-SGD variant for relational data with provable privacy guarantees. Experiments on fine-tuning text encoders over text-attributed network-structured relational data demonstrate the strong utility-privacy trade-offs of our approach. Our code is available at https://github.com/Graph-COM/Node_DP.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 54 of its 84 references that are in the map. First posted on arXiv 10 Jun 2025.
Builds on
54 of 84 references.
References · 54
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2017Inductive Representation Learning on Large Graphscited by 6,320
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
- 2016Deep Learning with Differential Privacycited by 2,728
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2020Graph Contrastive Learning with Augmentationscited by 1,208
- 2016Geometric deep learning: going beyond Euclidean datacited by 1,048
- 2019Publicly Available Clinical BERT Embeddingscited by 1,002
38 more
- 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 932
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
- 2015Image-based Recommendations on Styles and Substitutescited by 706
- 2017Renyi Differential Privacycited by 650
- 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 528
- 2015A Review of Relational Machine Learning for Knowledge Graphscited by 265
- 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 242
- 2018Modeling polypharmacy side effects with graph convolutional networkscited by 237
- 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 236
- 2019Differentially Private Learning with Adaptive Clippingcited by 220
- 2022LinkBERT: Pretraining Language Models with Document Linkscited by 211
- 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 204
- 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 171
- 2019Hypothesis Testing Interpretations and Renyi Differential Privacycited by 85
- 2018One-Shot Relational Learning for Knowledge Graphscited by 78
- 2021A Review on Graph Neural Network Methods in Financial Applicationscited by 78
- 2023SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learningcited by 73
- 2021Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Futurecited by 67
- 2020Locally Private Graph Neural Networkscited by 64
- 2022GAP: Differentially Private Graph Neural Networks with Aggregation Perturbationcited by 46
- 2022Composition of Differential Privacy & Privacy Amplification by Subsamplingcited by 45
- 2021Node-Level Differentially Private Graph Neural Networkscited by 43
- 2023Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applicationscited by 33
- 2021Releasing Graph Neural Networks with Differential Privacy Guaranteescited by 32
- 2022Differentially private training of residual networks with scale normalisationcited by 26
- 2020Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approachcited by 20
- 2014Characterizing the Sample Complexity of Private Learnerscited by 19
- 2021Private Graph Data Release: A Surveycited by 15
- 2022Differentially Private Community Detection for Stochastic Block Modelscited by 15
- 2023ProGAP: Progressive Graph Neural Networks with Differential Privacy Guaranteescited by 12
- 2023Differentially Private Decoupled Graph Convolutions for Multigranular Topology Protectioncited by 12
- 2023Preserving Node-level Privacy in Graph Neural Networkscited by 11
- 2022SoK: Differential Privacy on Graph-Structured Datacited by 10
- 2024On the Convergence of DP-SGD with Adaptive Clippingcited by 6
- 2025DC-SGD: Differentially Private SGD with Dynamic Clipping through Gradient Norm Distribution Estimationcited by 6
- 2023Pre-trained Neural Recommenders: A Transferable Zero-Shot Framework for Recommendation Systemscited by 3
- 2024Unified Mechanism-Specific Amplification by Subsampling and Group Privacy Amplificationcited by 3
- 2024Tight Group-Level DP Guarantees for DP-SGD with Sampling via Mixture of Gaussians Mechanismscited by 2
50 most cited of 54 All 54 (JSON)
Built on by
Not cited yet.