# Generating Synthetic Fair Syntax-agnostic Data by Learning and Distilling Fair Representation

arXiv: https://arxiv.org/abs/2408.10755 · HTML: https://mlsanity.com/paper/2408.10755/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Builds on (most cited first, 16 of 16)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [To be Robust or to be Fair: Towards Fairness in Adversarial Training](https://mlsanity.com/paper/2010.06121/) (2020)
- [Learning Debiased Representation via Disentangled Feature Augmentation](https://mlsanity.com/paper/2107.01372/) (2021)
- [General Instance Distillation for Object Detection](https://mlsanity.com/paper/2103.02340/) (2021)
- [Invariant Representations without Adversarial Training](https://mlsanity.com/paper/1805.09458/) (2018)
- [DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networks](https://mlsanity.com/paper/2110.12884/) (2021)
- [Fair Feature Distillation for Visual Recognition](https://mlsanity.com/paper/2106.04411/) (2021)
- [FairNeuron: Improving Deep Neural Network Fairness with Adversary Games on Selective Neurons](https://mlsanity.com/paper/2204.02567/) (2022)
- [The Devil is in the Data: Learning Fair Graph Neural Networks via Partial Knowledge Distillation](https://mlsanity.com/paper/2311.17373/) (2023)
- [A Variational Approach to Privacy and Fairness](https://mlsanity.com/paper/2006.06332/) (2020)
- [FairX: A comprehensive benchmarking tool for model analysis using fairness, utility, and explainability](https://mlsanity.com/paper/2406.14281/) (2024)

## Built on by (most cited first, 1 of 1)

- [Fair4Free: Generating High-fidelity Fair Synthetic Samples using Data Free Distillation](https://mlsanity.com/paper/2410.01423/) (2024)

## Abstract (arXiv)

Data Fairness is a crucial topic due to the recent wide usage of AI powered applications. Most of the real-world data is filled with human or machine biases and when those data are being used to train AI models, there is a chance that the model will reflect the bias in the training data. Existing bias-mitigating generative methods based on GANs, Diffusion models need in-processing fairness objectives and fail to consider computational overhead while choosing computationally-heavy architectures, which may lead to high computational demands, instability and poor optimization performance. To mitigate this issue, in this work, we present a fair data generation technique based on knowledge distillation, where we use a small architecture to distill the fair representation in the latent space. The idea of fair latent space distillation enables more flexible and stable training of Fair Generative Models (FGMs). We first learn a syntax-agnostic (for any data type) fair representation of the data, followed by distillation in the latent space into a smaller model. After distillation, we use the distilled fair latent space to generate high-fidelity fair synthetic data. While distilling, we employ quality loss (for fair distillation) and utility loss (for data utility) to ensure that the fairness and data utility characteristics remain in the distilled latent space. Our approaches show a 5%, 5% and 10% rise in performance in fairness, synthetic sample quality and data utility, respectively, than the state-of-the-art fair generative model.