# DecHW: Heterogeneous Decentralized Federated Learning Exploiting Second-Order Information

arXiv: https://arxiv.org/abs/2601.19938 · HTML: https://mlsanity.com/paper/2601.19938/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 16 Jan 2026.

## Builds on (most cited first, 11 of 11)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges](https://mlsanity.com/paper/2211.08413/) (2022)
- [Communication Efficient Distributed Optimization using an Approximate Newton-type Method](https://mlsanity.com/paper/1312.7853/) (2013)
- [BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning](https://mlsanity.com/paper/1905.06731/) (2019)
- [Decentralized Federated Averaging](https://mlsanity.com/paper/2104.11375/) (2021)
- [Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks](https://mlsanity.com/paper/1912.13163/) (2019)
- [Distributed Second Order Methods with Fast Rates and Compressed Communication](https://mlsanity.com/paper/2102.07158/) (2021)
- [Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors](https://mlsanity.com/paper/2405.11525/) (2024)
- [Decentralized Bayesian Learning over Graphs](https://mlsanity.com/paper/1905.10466/) (2019)
- [Coordination-free Decentralised Federated Learning on Complex Networks: Overcoming Heterogeneity](https://mlsanity.com/paper/2312.04504/) (2023)

## Abstract (arXiv)

Decentralized Federated Learning (DFL) is a serverless collaborative machine learning paradigm where devices collaborate directly with neighbouring devices to exchange model information for learning a generalized model. However, variations in individual experiences and different levels of device interactions lead to data and model initialization heterogeneities across devices. Such heterogeneities leave variations in local model parameters across devices that leads to slower convergence. This paper tackles the data and model heterogeneity by explicitly addressing the parameter level varying evidential credence across local models. A novel aggregation approach is introduced that captures these parameter variations in local models and performs robust aggregation of neighbourhood local updates. Specifically, consensus weights are generated via approximation of second-order information of local models on their local datasets. These weights are utilized to scale neighbourhood updates before aggregating them into global neighbourhood representation. In extensive experiments with computer vision tasks, the proposed approach shows strong generalizability of local models at reduced communication costs.