mlsanityData to 7 Oct 2026

FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices

arXiv 2604.25421 ↗First posted 28 Apr 2026cs.LG cs.AIcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

Federated fine-tuning provides a practical route to adapt large language models (LLMs) on edge devices without centralizing private data. However, in mobile deployments, the training wall-clock is often dominated by straggler-limited uplink communication under heterogeneous bandwidth, intermittent participation, and non-IID client data. Although parameter-efficient fine-tuning (PEFT) methods such as LoRA and QLoRA reduce local memory and trainable parameters, repeated transmission of adapter updates remains a major bottleneck. We propose Fed-FSTQ, a semantic-sensitivity-aware communication-control primitive for communication-efficient federated LLM fine-tuning. Fed-FSTQ uses a lightweight token-level Fisher proxy to estimate semantic sensitivity, couples token-guided sparsification with mixed-precision adapter-update quantization, and allocates higher communication fidelity to semantically load-bearing evidence while suppressing redundant transmission. The method is drop-in compatible with standard federated PEFT pipelines and requires no change to the server aggregation rule. Experiments on multilingual QA and medical QA under non-IID partitions show that Fed-FSTQ reduces cumulative uplink traffic required to reach a fixed quality threshold by 46-fold relative to a Fed-LoRA baseline and improves straggler-limited wall-clock time-to-accuracy by 52%. Under the corrected Controlled LTE-20Mbps accounting, Fed-FSTQ reduces per-round time from 414.60s to 67.29s and reduces per-round energy from 839.20J to 146.28J, yielding a 6.16-fold speedup. On NVIDIA Jetson-class edge devices, Fisher-guided token reduction also yields up to a 1.55-fold inference speedup, demonstrating deployability under tight resource constraints.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 48 of its 69 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

48 of 69 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2019Advances and Open Problems in Federated Learningcited by 3,021
  11. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  12. 2016Deep Learning with Differential Privacycited by 2,682
36 more
  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  11. 2020Adaptive Federated Optimizationcited by 1,036
  12. 2022Token Merging: Your ViT But Fastercited by 995
  13. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  14. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  15. 2019Towards Federated Learning at Scale: System Designcited by 826
  16. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  17. 2018LEAF: A Benchmark for Federated Settingscited by 780
  18. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  19. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  20. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  21. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  22. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  23. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  24. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  25. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  26. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  27. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  28. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
  29. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
  30. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  31. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 181
  32. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  33. 2018Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communicationcited by 51
  34. 2024FedBAT: Communication-Efficient Federated Learning via Learnable Binarizationcited by 9
  35. 2024Adaptive Parameter-Efficient Federated Fine-Tuning on Heterogeneous Devicescited by 5
  36. 2025FedLoDrop: Federated LoRA with Dropout for Generalized LLM Fine-tuningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning
  2. 2026AlignFed: Alignment-Aware Asynchronous Federated Fine-Tuning for Large Language Models in Heterogeneous Edge Environments
  3. 2025FedSODA: Federated Fine-tuning of LLMs via Similarity Group Pruning and Orchestrated Distillation Alignment
  4. 2025FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
  5. 2026Federated Inference for Heterogeneous LLM Communication and Collaboration
  6. 2026CFLoRA: Federated Fine-tuning of LLMs with Complementary Factors for Error-free Aggregation
  7. 2026TSFLora: Token-Compressed Split Fine-Tuning for Wireless Edge Networks
  8. 2025FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA