# Deep Equilibrium Models Meet Federated Learning

arXiv: https://arxiv.org/abs/2305.18646 · HTML: https://mlsanity.com/paper/2305.18646/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 14 of 14)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Data-Free Knowledge Distillation for Heterogeneous Federated Learning](https://mlsanity.com/paper/2105.10056/) (2021)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [FetchSGD: Communication-Efficient Federated Learning with Sketching](https://mlsanity.com/paper/2007.07682/) (2020)
- [Multiscale Deep Equilibrium Models](https://mlsanity.com/paper/2006.08656/) (2020)
- [Parameterized Knowledge Transfer for Personalized Federated Learning](https://mlsanity.com/paper/2111.02862/) (2021)
- [Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective](https://mlsanity.com/paper/2004.04314/) (2020)
- [Asynchronous Federated Learning on Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2109.04269/) (2021)
- [Federated Learning: A Signal Processing Perspective](https://mlsanity.com/paper/2103.17150/) (2021)
- [Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certification](https://mlsanity.com/paper/2106.01453/) (2021)
- [Connections between Deep Equilibrium and Sparse Representation Models with Application to Hyperspectral Image Denoising](https://mlsanity.com/paper/2203.15901/) (2022)

## Built on by (most cited first, 3 of 3)

- [Positive concave deep equilibrium models](https://mlsanity.com/paper/2402.04029/) (2024)
- [Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach](https://mlsanity.com/paper/2602.03297/) (2026)
- [Towards Resource-Efficient Federated Learning in Industrial IoT for Multivariate Time Series Analysis](https://mlsanity.com/paper/2411.03996/) (2024)

## Abstract (arXiv)

In this study the problem of Federated Learning (FL) is explored under a new perspective by utilizing the Deep Equilibrium (DEQ) models instead of conventional deep learning networks. We claim that incorporating DEQ models into the federated learning framework naturally addresses several open problems in FL, such as the communication overhead due to the sharing large models and the ability to incorporate heterogeneous edge devices with significantly different computation capabilities. Additionally, a weighted average fusion rule is proposed at the server-side of the FL framework to account for the different qualities of models from heterogeneous edge devices. To the best of our knowledge, this study is the first to establish a connection between DEQ models and federated learning, contributing to the development of an efficient and effective FL framework. Finally, promising initial experimental results are presented, demonstrating the potential of this approach in addressing challenges of FL.