mlsanityData to 14 Oct 2026

A Federated Learning Benchmark on Tabular Data: Comparing Tree-Based Models and Neural Networks

arXiv 2405.02074 ↗First posted 3 May 2024cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has lately gained traction as it addresses how machine learning models train on distributed datasets. FL was designed for parametric models, namely Deep Neural Networks (DNNs).Thus, it has shown promise on image and text tasks. However, FL for tabular data has received little attention. Tree-Based Models (TBMs) have been considered to perform better on tabular data and they are starting to see FL integrations. In this study, we benchmark federated TBMs and DNNs for horizontal FL, with varying data partitions, on 10 well-known tabular datasets. Our novel benchmark results indicates that current federated boosted TBMs perform better than federated DNNs in different data partitions. Furthermore, a federated XGBoost outperforms all other models. Lastly, we find that federated TBMs perform better than federated parametric models, even when increasing the number of clients significantly.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 33 of its 56 references that are in the map. First posted on arXiv 3 May 2024.

Builds on

33 of 56 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  8. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  9. 2018LEAF: A Benchmark for Federated Settingscited by 792
  10. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  11. 2019TabNet: Attentive Interpretable Tabular Learningcited by 648
  12. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
21 more
  1. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  2. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  3. 2021Towards Personalized Federated Learningcited by 464
  4. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
  5. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  6. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  7. 2018AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networkscited by 320
  8. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
  9. 2019Neural Oblivious Decision Ensembles for Deep Learning on Tabular Datacited by 203
  10. 2022Why do tree-based models still outperform deep learning on tabular data?cited by 199
  11. 2019SecureBoost: A Lossless Federated Learning Frameworkcited by 176
  12. 2019Sparse Sequence-to-Sequence Modelscited by 170
  13. 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
  14. 2022NVIDIA FLARE: Federated Learning from Simulation to Real-Worldcited by 80
  15. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
  16. 2018Deep Neural Decision Treescited by 72
  17. 2019Federated Forestcited by 49
  18. 2020FederBoost: Private Federated Learning for GBDTcited by 27
  19. 2022FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulationscited by 27
  20. 2019The Tradeoff Between Privacy and Accuracy in Anomaly Detection Using Federated XGBoostcited by 13
  21. 2020A Comparative Study of Machine Learning Models for Tabular Data Through Challenge of Monitoring Parkinson's Disease Progression Using Voice Recordingscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 1.

3 papers.

  1. 2024FedCVD: The First Real-World Federated Learning Benchmark on Cardiovascular Disease Datacited by 2
  2. 2024MonoSparse-CAM: Efficient Tree Model Processing via Monotonicity and Sparsity in CAMscited by 0
  3. 2026Towards Explainable Federated Learning: Understanding the Impact of Differential Privacycited by 0