mlsanityData to 14 Oct 2026

Ampere: Communication-Efficient and High-Accuracy Split Federated Learning

arXiv 2507.07130 ↗First posted 8 Jul 2025cs.DC cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

A Federated Learning (FL) system collaboratively trains neural networks across devices and a server but is limited by significant on-device computation costs. Split Federated Learning (SFL) systems mitigate this by offloading a block of layers of the network from the device to a server. However, in doing so, it introduces large communication overheads due to frequent exchanges of intermediate activations and gradients between devices and the server and reduces model accuracy for non-IID data. We propose Ampere, a novel collaborative training system that simultaneously minimizes on-device computation and device-server communication while improving model accuracy. Unlike SFL, which uses a global loss by iterative end-to-end training, Ampere develops unidirectional inter-block training to sequentially train the device and server blocks with a local loss, eliminating the transfer of gradients. A lightweight auxiliary network generation method decouples training between the device and server, reducing frequent intermediate exchanges to a single transfer, which significantly reduces the communication overhead. Ampere mitigates the impact of data heterogeneity by consolidating activations generated by the trained device block to train the server block, in contrast to SFL, which trains on device-specific, non-IID activations. Extensive experiments on multiple CNNs and Transformers show that, compared to state-of-the-art SFL baseline systems, Ampere (i) improves model accuracy by up to 11.70 percentage points while training up to 18.6x faster, (ii) incurs up to 911x lower device-server communication overhead and up to 14.5x lower on-device computation, and (iii) reduces standard deviation of accuracy by 71.13% for various non-IID degrees highlighting superior performance when faced with heterogeneous data. Ampere is available from https://github.com/blessonvar/Ampere.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

47 of 69 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  11. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  12. 2016Deep Learning with Differential Privacycited by 2,728
35 more
  1. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  2. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  3. 2019Searching for MobileNetV3cited by 1,994
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  6. 2014How transferable are features in deep neural networks?cited by 1,225
  7. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  9. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  10. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  11. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 564
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  13. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 468
  14. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  15. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 395
  16. 2018Sparsified SGD with Memorycited by 376
  17. 2020SplitFed: When Federated Learning Meets Split Learningcited by 310
  18. 2018Distributed learning of deep neural network over multiple agentscited by 279
  19. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  20. 2019Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computingcited by 228
  21. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 195
  22. 2017Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsingcited by 109
  23. 2019BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systemscited by 95
  24. 2023Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networkscited by 95
  25. 2019Decoupled Greedy Learning of CNNscited by 78
  26. 2021FedAdapt: Adaptive Offloading for IoT Devices in Federated Learningcited by 26
  27. 2018Training Neural Networks Using Features Replaycited by 23
  28. 2023Reducing Communication for Split Learning by Randomized Top-k Sparsificationcited by 22
  29. 2022SplitGP: Achieving Both Generalization and Personalization in Federated Learningcited by 20
  30. 2020Back-and-Forth prediction for deep tensor compressioncited by 16
  31. 2024ParallelSFL: A Novel Split Federated Learning Framework Tackling Heterogeneity Issuescited by 10
  32. 2022C3-SL: Circular Convolution-Based Batch-Wise Compression for Communication-Efficient Split Learningcited by 8
  33. 2023EcoFed: Efficient Communication for DNN Partitioning-based Federated Learningcited by 5
  34. 2024Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splittingcited by 4
  35. 2022PiPar: Pipeline Parallelism for Collaborative Machine Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizationscited by 0
  2. 2026When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimationcited by 0