mlsanityData to 14 Oct 2026

Traversal Learning: A Lossless And Efficient Distributed Learning Framework

arXiv 2504.07471 ↗First posted 10 Apr 2025cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce Traversal Learning (TL), a novel approach designed to address the problem of decreased quality encountered in popular distributed learning (DL) paradigms such as Federated Learning (FL), Split Learning (SL), and SplitFed Learning (SFL). Traditional FL experiences from an accuracy drop during aggregation due to its averaging function, while SL and SFL face increased loss due to the independent gradient updates on each split network. TL adopts a unique strategy where the model traverses the nodes during forward propagation (FP) and performs backward propagation (BP) on the orchestrator, effectively implementing centralized learning (CL) principles within a distributed environment. The orchestrator is tasked with generating virtual batches and planning the sequential node visits of the model during FP, aligning them with the ordered index of the data within these batches. We conducted experiments on six datasets representing diverse characteristics across various domains. Our evaluation demonstrates that TL is on par with classic CL approaches in terms of accurate inference, thereby offering a viable and robust solution for DL tasks. TL outperformed other DL methods and improved accuracy by 7.85% for independent and identically distributed (IID) datasets, macro F1-score by 1.06% for non-IID datasets, accuracy by 2.60% for text classification, and AUC by 3.88% and 4.54% for medical and financial datasets, respectively. By effectively preserving data privacy while maintaining performance, TL represents a significant advancement in DL methodologies. The implementation of TL is available at https://github.com/neouly-inc/Traversal-Learning

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 77 references that are in the map. First posted on arXiv 10 Apr 2025.

Builds on

41 of 77 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  7. 2020Federated Learning with Matched Averagingcited by 564
  8. 2021Towards Personalized Federated Learningcited by 464
  9. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 395
  10. 2020Adaptive Personalized Federated Learningcited by 358
  11. 2019Robust Aggregation for Federated Learningcited by 317
  12. 2020SplitFed: When Federated Learning Meets Split Learningcited by 310
29 more
  1. 2018Distributed learning of deep neural network over multiple agentscited by 279
  2. 2019Asynchronous Federated Optimizationcited by 267
  3. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  4. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 175
  5. 2021Decentralized Federated Averagingcited by 114
  6. 2021Distributionally Robust Federated Averagingcited by 104
  7. 2021Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devicescited by 56
  8. 2023A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Taskscited by 41
  9. 2022On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyondcited by 38
  10. 2021Emerging Trends in Federated Learning: From Model Fusion to Federated X Learningcited by 35
  11. 2020Federated Transfer Learning: concept and applicationscited by 32
  12. 2021Benchmarking emergency department triage prediction models with machine learning and large public electronic health recordscited by 31
  13. 2020FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networkscited by 23
  14. 2021Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learningcited by 23
  15. 2019Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updatingcited by 22
  16. 2021Differential Privacy Meets Federated Learning under Communication Constraintscited by 16
  17. 2021A Non-parametric View of FedAvg and FedProx: Beyond Stationary Pointscited by 13
  18. 2023MergeSFL: Split Federated Learning with Feature Merging and Batch Size Regulationcited by 13
  19. 2022Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropoutcited by 12
  20. 2022SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable Communicationscited by 11
  21. 2024Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Surveycited by 10
  22. 2020Scalable Privacy-Preserving Distributed Learningcited by 9
  23. 2017Dropping Activation Outputs with Localized First-layer Deep Network for Enhancing User Privacy and Data Securitycited by 8
  24. 2024Privacy-Preserving Autoencoder for Collaborative Object Detectioncited by 5
  25. 2022Federated Learning Aggregation: New Robust Algorithms with Guaranteescited by 4
  26. 2022Split Learning without Local Weight Sharing to Enhance Client-side Data Privacycited by 4
  27. 2023FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropoutcited by 4
  28. 2023Rethinking Client Drift in Federated Learning: A Logit Perspectivecited by 4
  29. 2024Optimizing Split Points for Error-Resilient SplitFed Learningcited by 1

Built on by

Not cited yet.