# Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation

arXiv: https://arxiv.org/abs/2410.07662 · HTML: https://mlsanity.com/paper/2410.07662/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training](https://mlsanity.com/paper/2305.14342/) (2023)
- [FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learning](https://mlsanity.com/paper/2206.08829/) (2022)
- [DONE: Distributed Approximate Newton-type Method for Federated Edge Learning](https://mlsanity.com/paper/2012.05625/) (2020)
- [Fed-Sophia: A Communication-Efficient Second-Order Federated Learning Algorithm](https://mlsanity.com/paper/2406.06655/) (2024)
- [Machine Learning Empowered Resource Allocation in IRS Aided MISO-NOMA Networks](https://mlsanity.com/paper/2103.11791/) (2021)
- [LP-BFGS attack: An adversarial attack based on the Hessian with limited pixels](https://mlsanity.com/paper/2210.15446/) (2022)

## Built on by (most cited first, 1 of 1)

- [Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems](https://mlsanity.com/paper/2506.22374/) (2025)

## Abstract (arXiv)

Second-order federated learning (FL) algorithms offer faster convergence than their first-order counterparts by leveraging curvature information. However, they are hindered by high computational and storage costs, particularly for large-scale models. Furthermore, the communication overhead associated with large models and digital transmission exacerbates these challenges, causing communication bottlenecks. In this work, we propose a scalable second-order FL algorithm using a sparse Hessian estimate and leveraging over-the-air aggregation, making it feasible for larger models. Our simulation results demonstrate more than 67% of communication resources and energy savings compared to other first and second-order baselines.