# ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge

arXiv: https://arxiv.org/abs/2601.13824 · HTML: https://mlsanity.com/paper/2601.13824/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Builds on (most cited first, 26 of 26)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment](https://mlsanity.com/paper/2312.12148/) (2023)
- [FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction](https://mlsanity.com/paper/2203.11751/) (2022)
- [Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks](https://mlsanity.com/paper/2303.15991/) (2023)
- [Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence](https://mlsanity.com/paper/2305.03010/) (2023)
- [SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2407.00952/) (2024)
- [Communication-Efficient Adaptive Federated Learning](https://mlsanity.com/paper/2205.02719/) (2022)
- [Fine-Tuning is Fine, if Calibrated](https://mlsanity.com/paper/2409.16223/) (2024)
- [FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching](https://mlsanity.com/paper/2008.04975/) (2020)
- [Toward Edge General Intelligence with Multiple-Large Language Model (Multi-LLM): Architecture, Trust, and Orchestration](https://mlsanity.com/paper/2507.00672/) (2025)
- [Information Leakage from Embedding in Large Language Models](https://mlsanity.com/paper/2405.11916/) (2024)
- [Hierarchical Federated Learning with Multi-Timescale Gradient Correction](https://mlsanity.com/paper/2409.18448/) (2024)
- [Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Experts](https://mlsanity.com/paper/2508.00234/) (2025)
- [Split Fine-Tuning for Large Language Models in Wireless Networks](https://mlsanity.com/paper/2501.09237/) (2025)
- [FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learning](https://mlsanity.com/paper/2405.11811/) (2024)
- [SplitLLM: Hierarchical Split Learning for Large Language Model over Wireless Network](https://mlsanity.com/paper/2501.13318/) (2025)
- [eFedLLM: Efficient LLM Inference Based on Federated Learning](https://mlsanity.com/paper/2411.16003/) (2024)
- [LEAP: Optimization Hierarchical Federated Learning on Non-IID Data with Coalition Formation Game](https://mlsanity.com/paper/2405.00579/) (2024)
- [Adaptive UAV-Assisted Hierarchical Federated Learning: Optimizing Energy, Latency, and Resilience for Dynamic Smart IoT](https://mlsanity.com/paper/2503.06145/) (2025)
- [Deploying Large AI Models on Resource-Limited Devices with Split Federated Learning](https://mlsanity.com/paper/2504.09114/) (2025)
- [HFL-FlowLLM: Large Language Models for Network Traffic Flow Classification in Heterogeneous Federated Learning](https://mlsanity.com/paper/2511.14199/) (2025)

## Abstract (arXiv)

Training large language models (LLMs) at the network edge faces fundamental challenges arising from device resource constraints, severe data heterogeneity, and heightened privacy risks. To address these challenges, we propose ELSA (Efficient LLM-centric Split Aggregation), a novel framework that systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks. ELSA introduces three key innovations. First, it employs a task-agnostic, behavior-aware client clustering mechanism that constructs semantic fingerprints using public probe inputs and symmetric Kullback-Leibler (KL) divergence, augmented by prediction-consistency trust scoring and latency-aware edge assignment to jointly mitigate data heterogeneity, device unreliability, and communication constraints. Second, it employs a resource-aware dynamic model splitting strategy to adaptively partition the LLM into three segments across clients and edge servers, with the cloud used only for adapter aggregation, enabling an effective balance between on-device computation cost and global convergence stability. Third, it incorporates a lightweight communication scheme based on computational sketches combined with semantic subspace orthogonal perturbation (SS-OP) to reduce communication overhead while mitigating privacy leakage during model exchanges across the network. Extensive experiments across diverse NLP tasks demonstrate that ELSA consistently outperforms state-of-the-art baselines in terms of adaptability, convergence behavior, and robustness, establishing a scalable and privacy-aware solution for edge-side LLM fine-tuning under resource constraints.