# Package for Fast ABC-Boost

arXiv: https://arxiv.org/abs/2207.08770 · HTML: https://mlsanity.com/paper/2207.08770/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 27 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 5 of 5)

- [Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems](https://mlsanity.com/paper/2003.05622/) (2020)
- [GCWSNet: Generalized Consistent Weighted Sampling for Scalable and Accurate Training of Neural Networks](https://mlsanity.com/paper/2201.02283/) (2022)
- [Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising](https://mlsanity.com/paper/2201.05500/) (2022)
- [Integrity Authentication in Tree Models](https://mlsanity.com/paper/2205.15444/) (2022)
- [Fast ABC-Boost: A Unified Framework for Selecting the Base Class in Multi-Class Classification](https://mlsanity.com/paper/2205.10927/) (2022)

## Built on by (most cited first, 2 of 2)

- [Boost CTR Prediction for New Advertisements via Modeling Visual Content](https://mlsanity.com/paper/2209.11727/) (2022)
- [Online Gradient Boosting Decision Tree: In-Place Updates for Efficient Adding/Deleting Data](https://mlsanity.com/paper/2502.01634/) (2025)

## Abstract (arXiv)

This report presents the open-source package which implements the series of our boosting works in the past years. In particular, the package includes mainly three lines of techniques, among which the following two are already the standard implementations in popular boosted tree platforms: (i) The histogram-based (feature-binning) approach makes the tree implementation convenient and efficient. In Li et al (2007), a simple fixed-length adaptive binning algorithm was developed. In this report, we demonstrate that such a simple algorithm is still surprisingly effective compared to more sophisticated variants in popular tree platforms. (ii) The explicit gain formula in Li (20010) for tree splitting based on second-order derivatives of the loss function typically improves, often considerably, over the first-order methods. Although the gain formula in Li (2010) was derived for logistic regression loss, it is a generic formula for loss functions with second-derivatives. For example, the open-source package also includes Lp regression for p≥ 1. The main contribution of this package is the ABC-Boost (adaptive base class boosting) for multi-class classification. The initial work in Li (2008) derived a new set of derivatives of the classical multi-class logistic regression by specifying a "base class". The accuracy can be substantially improved if the base class is chosen properly. The major technical challenge is to design a search strategy to select the base class. The prior published works implemented an exhaustive search procedure to find the base class which is computationally too expensive. Recently, a new report (Li and Zhao, 20022) presents a unified framework of "Fast ABC-Boost" which allows users to efficiently choose the proper search space for the base class. The package provides interfaces for linux, windows, mac, matlab, R, python.