# ALIGN-FL: Architecture-independent Learning through Invariant Generative component sharing in Federated Learning

arXiv: https://arxiv.org/abs/2512.13316 · HTML: https://mlsanity.com/paper/2512.13316/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 15 Dec 2025.

## Builds on (most cited first, 17 of 17)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [Differentially Private Learning with Adaptive Clipping](https://mlsanity.com/paper/1905.03871/) (2019)
- [Numerical Composition of Differential Privacy](https://mlsanity.com/paper/2106.02848/) (2021)
- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data](https://mlsanity.com/paper/2404.01413/) (2024)
- [Nemotron-4 340B Technical Report](https://mlsanity.com/paper/2406.11704/) (2024)
- [GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators](https://mlsanity.com/paper/2006.08265/) (2020)
- [Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop](https://mlsanity.com/paper/2311.16822/) (2023)
- [Towards Theoretical Understandings of Self-Consuming Generative Models](https://mlsanity.com/paper/2402.11778/) (2024)
- [FedCiR: Client-Invariant Representation Learning for Federated Non-IID Features](https://mlsanity.com/paper/2308.15786/) (2023)
- [PaDPaF: Partial Disentanglement with Partially-Federated GANs](https://mlsanity.com/paper/2212.03836/) (2022)
- [Differentially Private Synthetic Data Generation via Lipschitz-Regularised Variational Autoencoders](https://mlsanity.com/paper/2304.11336/) (2023)

## Abstract (arXiv)

We present ALIGN-FL, a novel approach to distributed learning that addresses the challenge of learning from highly disjoint data distributions through selective sharing of generative components. Instead of exchanging full model parameters, our framework enables privacy-preserving learning by transferring only generative capabilities across clients, while the server performs global training using synthetic samples. Through complementary privacy mechanisms: DP-SGD with adaptive clipping and Lipschitz regularized VAE decoders and a stateful architecture supporting heterogeneous clients, we experimentally validate our approach on MNIST and Fashion-MNIST datasets with cross-domain outliers. Our analysis demonstrates that both privacy mechanisms effectively map sensitive outliers to typical data points while maintaining utility in extreme Non-IID scenarios typical of cross-silo collaborations. Index Terms: Client-invariant Learning, Federated Learning (FL), Privacy-preserving Generative Models, Non-Independent and Identically Distributed (Non-IID), Heterogeneous Architectures