# Photonic Differential Privacy with Direct Feedback Alignment

arXiv: https://arxiv.org/abs/2106.03645 · HTML: https://mlsanity.com/paper/2106.03645/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [Subsampled R'enyi Differential Privacy and Analytical Moments Accountant](https://mlsanity.com/paper/1808.00087/) (2018)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures](https://mlsanity.com/paper/2006.12878/) (2020)
- [Training of photonic neural networks through in situ backpropagation](https://mlsanity.com/paper/1805.09943/) (2018)
- [Differentially Private Mixture of Generative Neural Networks](https://mlsanity.com/paper/1709.04514/) (2017)
- [Three Variants of Differential Privacy: Lossless Conversion and Applications](https://mlsanity.com/paper/2008.06529/) (2020)
- [High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial](https://mlsanity.com/paper/1412.4031/) (2014)
- [Towards Practical Differential Privacy for SQL Queries](https://mlsanity.com/paper/1706.09479/) (2017)
- [Removing Disparate Impact of Differentially Private Stochastic Gradient Descent on Model Accuracy](https://mlsanity.com/paper/2003.03699/) (2020)
- [Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment](https://mlsanity.com/paper/2012.06373/) (2020)
- [MVG Mechanism: Differential Privacy under Matrix-Valued Query](https://mlsanity.com/paper/1801.00823/) (2018)
- [Differentially Private Deep Learning with Direct Feedback Alignment](https://mlsanity.com/paper/2010.03701/) (2020)
- [Kernel computations from large-scale random features obtained by Optical Processing Units](https://mlsanity.com/paper/1910.09880/) (2019)
- [Backpropagation through nonlinear units for all-optical training of neural networks](https://mlsanity.com/paper/1912.12256/) (2019)

## Built on by (most cited first, 5 of 5)

- [Nonlinear optical encoding enabled by recurrent linear scattering](https://mlsanity.com/paper/2307.08558/) (2023)
- [Self-Supervised Pretraining for Differentially Private Learning](https://mlsanity.com/paper/2206.07125/) (2022)
- [Scaling Laws Beyond Backpropagation](https://mlsanity.com/paper/2210.14593/) (2022)
- [Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignment](https://mlsanity.com/paper/2409.12965/) (2024)
- [DFA-GNN: Forward Learning of Graph Neural Networks by Direct Feedback Alignment](https://mlsanity.com/paper/2406.02040/) (2024)

## Abstract (arXiv)

Optical Processing Units (OPUs) -- low-power photonic chips dedicated to large scale random projections -- have been used in previous work to train deep neural networks using Direct Feedback Alignment (DFA), an effective alternative to backpropagation. Here, we demonstrate how to leverage the intrinsic noise of optical random projections to build a differentially private DFA mechanism, making OPUs a solution of choice to provide a private-by-design training. We provide a theoretical analysis of our adaptive privacy mechanism, carefully measuring how the noise of optical random projections propagates in the process and gives rise to provable Differential Privacy. Finally, we conduct experiments demonstrating the ability of our learning procedure to achieve solid end-task performance.