Minimal Model Structure Analysis for Input Reconstruction in Federated Learning

acfl proposed a distributed acml framework where every distributed worker owns a complete copy of global model and their own data. The training is occurred locally, which assures no direct transmission of training data. However, the recent work citepzhu2019deep demonstrated that input data from a neural network may be reconstructed only using knowledge of gradients of that network, which completely breached the promise of acfl and sabotaged the user privacy. In this work, we aim to further explore the theoretical limits of reconstruction, speedup and stabilize the reconstruction procedure. We show that a single input may be reconstructed with the analytical form, regardless of network depth using a fully-connected neural network with one hidden node. Then we generalize this result to a gradient averaged over batches of size B. In this case, the full batch can be reconstructed if the number of hidden units exceeds B. For a accnn, the number of required kernels in convolutional layers is decided by multiple factors, e.g., padding, kernel and stride size, etc. We require the number of kernels h≥ (fracddprime)2C, where we define d as input width, dprime as output width after convolutional layer, and C as channel number of input. We validate our observation and demonstrate the improvements using bio-medical (fMRI, acwbc) and benchmark data (MNIST, Kuzushiji-MNIST, CIFAR100, ImageNet and face images).