mlsanityData to 14 Oct 2026

The Devil is in the Data: Learning Fair Graph Neural Networks via Partial Knowledge Distillation

arXiv 2311.17373 ↗First posted 29 Nov 2023cs.LG cs.CY cs.SIcited by 16

On the map: Knowledge distillation → Graph neural network distillation

Abstract (arXiv)

Graph neural networks (GNNs) are being increasingly used in many high-stakes tasks, and as a result, there is growing attention on their fairness recently. GNNs have been shown to be unfair as they tend to make discriminatory decisions toward certain demographic groups, divided by sensitive attributes such as gender and race. While recent works have been devoted to improving their fairness performance, they often require accessible demographic information. This greatly limits their applicability in real-world scenarios due to legal restrictions. To address this problem, we present a demographic-agnostic method to learn fair GNNs via knowledge distillation, namely FairGKD. Our work is motivated by the empirical observation that training GNNs on partial data (i.e., only node attributes or topology data) can improve their fairness, albeit at the cost of utility. To make a balanced trade-off between fairness and utility performance, we employ a set of fairness experts (i.e., GNNs trained on different partial data) to construct the synthetic teacher, which distills fairer and informative knowledge to guide the learning of the GNN student. Experiments on several benchmark datasets demonstrate that FairGKD, which does not require access to demographic information, significantly improves the fairness of GNNs by a large margin while maintaining their utility.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Graph neural network distillation, one of 12 directions in Knowledge distillation. Builds on 29 of its 42 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

29 of 42 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  7. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  8. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  9. 2019Graph Neural Networks for Social Recommendationcited by 714
  10. 2020Deep Graph Contrastive Representation Learningcited by 539
  11. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 364
  12. 2021Graph Neural Network for Traffic Forecasting: A Surveycited by 261
17 more
  1. 2020Fairness without Demographics through Adversarially Reweighted Learningcited by 251
  2. 2020Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Informationcited by 168
  3. 2017Data Decisions and Theoretical Implications when Adversarially Learning Fair Representationscited by 165
  4. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 152
  5. 2019Identifying and Correcting Label Bias in Machine Learningcited by 146
  6. 2019Compositional Fairness Constraints for Graph Embeddingscited by 126
  7. 2021EDITS: Modeling and Mitigating Data Bias for Graph Neural Networkscited by 92
  8. 2020Alleviating the Inconsistency Problem of Applying Graph Neural Network to Fraud Detectioncited by 82
  9. 2022Fairness in Graph Mining: A Surveycited by 76
  10. 2021FairDrop: Biased Edge Dropout for Enhancing Fairness in Graph Representation Learningcited by 68
  11. 2022Improving Fairness in Graph Neural Networks via Mitigating Sensitive Attribute Leakagecited by 61
  12. 2022Learning Fair Node Representations with Graph Counterfactual Fairnesscited by 58
  13. 2021Towards Fair Classifiers Without Sensitive Attributes: Exploring Biases in Related Featurescited by 45
  14. 2020All of the Fairness for Edge Prediction with Optimal Transportcited by 36
  15. 2022FMP: Toward Fair Graph Message Passing against Topology Biascited by 35
  16. 2022FairEGM: Fair Link Prediction and Recommendation via Emulated Graph Modificationcited by 15
  17. 2022Learning Fair Models without Sensitive Attributes: A Generative Approachcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 6; 2026: 3.

16 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2023RELIANT: Fair Knowledge Distillation for Graph Neural Networks
  2. 2024Unbiased GNN Learning via Fairness-Aware Subgraph Diffusion
  3. 2024FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently
  4. 2026FairGU: Fairness-aware Graph Unlearning in Social Networks
  5. 2025Improving Group Fairness in Knowledge Distillation via Laplace Approximation of Early Exits
  6. 2025FROG: Fair Removal on Graphs
  7. 2023Knowledge Distillation on Graphs: A Survey
  8. 2024What is Left After Distillation? How Knowledge Transfer Impacts Fairness and Bias