# SecureBoost+: Large Scale and High-Performance Vertical Federated Gradient Boosting Decision Tree

arXiv: https://arxiv.org/abs/2110.10927 · HTML: https://mlsanity.com/paper/2110.10927/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 21 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Secure Federated Matrix Factorization](https://mlsanity.com/paper/1906.05108/) (2019)
- [Practical Federated Gradient Boosting Decision Trees](https://mlsanity.com/paper/1911.04206/) (2019)
- [A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression](https://mlsanity.com/paper/1912.00513/) (2019)
- [When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Control](https://mlsanity.com/paper/2008.08753/) (2020)
- [FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Training](https://mlsanity.com/paper/2008.10838/) (2020)
- [Additively Homomorphical Encryption based Deep Neural Network for Asymmetrically Collaborative Machine Learning](https://mlsanity.com/paper/2007.06849/) (2020)
- [GBDT-MO: Gradient Boosted Decision Trees for Multiple Outputs](https://mlsanity.com/paper/1909.04373/) (2019)
- [TitAnt: Online Real-time Transaction Fraud Detection in Ant Financial](https://mlsanity.com/paper/1906.07407/) (2019)
- [An Experimental Evaluation of Large Scale GBDT Systems](https://mlsanity.com/paper/1907.01882/) (2019)

## Built on by (most cited first, 2 of 2)

- [Networked Information Aggregation for Binary Classification](https://mlsanity.com/paper/2605.01082/) (2026)
- [Federated Imputation under Heterogeneous Feature Spaces](https://mlsanity.com/paper/2605.16099/) (2026)

## Abstract (arXiv)

Gradient boosting decision tree (GBDT) is an ensemble machine learning algorithm, which is widely used in industry, due to its good performance and easy interpretation. Due to the problem of data isolation and the requirement of privacy, many works try to use vertical federated learning to train machine learning models collaboratively with privacy guarantees between different data owners. SecureBoost is one of the most popular vertical federated learning algorithms for GBDT. However, in order to achieve privacy preservation, SecureBoost involves complex training procedures and time-consuming cryptography operations. This causes SecureBoost to be slow to train and does not scale to large scale data. In this work, we propose SecureBoost+, a large-scale and high-performance vertical federated gradient boosting decision tree framework. SecureBoost+ is secure in the semi-honest model, which is the same as SecureBoost. SecureBoost+ can be scaled up to tens of millions of data samples easily. SecureBoost+ achieves high performance through several novel optimizations for SecureBoost, including ciphertext operation optimization, the introduction of new training mechanisms, and multi-classification training optimization. The experimental results show that SecureBoost+ is 6-35x faster than SecureBoost, but with the same accuracy and can be scaled up to tens of millions of data samples and thousands of feature dimensions.