# CatBoost: unbiased boosting with categorical features

arXiv: https://arxiv.org/abs/1706.09516 · HTML: https://mlsanity.com/paper/1706.09516/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2017.

## Built on by (most cited first, 30 of 31)

- [Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Data](https://mlsanity.com/paper/2507.03971/) (2025)
- [An Open Multilingual System for Scoring Readability of Wikipedia](https://mlsanity.com/paper/2406.01835/) (2024)
- [Stochastic Configuration Machines for Industrial Artificial Intelligence](https://mlsanity.com/paper/2308.13570/) (2023)
- [Gradient Boosting Trees and Large Language Models for Tabular Data Few-Shot Learning](https://mlsanity.com/paper/2411.04324/) (2024)
- [Fair multilingual vandalism detection system for Wikipedia](https://mlsanity.com/paper/2306.01650/) (2023)
- [The Impact of Feature Scaling In Machine Learning: Effects on Regression and Classification Tasks](https://mlsanity.com/paper/2506.08274/) (2025)
- [On the Importance of Image Encoding in Automated Chest X-Ray Report Generation](https://mlsanity.com/paper/2211.13465/) (2022)
- [From Decision Trees to Boolean Logic: A Fast and Unified SHAP Algorithm](https://mlsanity.com/paper/2511.09376/) (2025)
- [Financial Transaction Retrieval and Contextual Evidence for Knowledge-Grounded Reasoning](https://mlsanity.com/paper/2603.15459/) (2026)
- [Quadrature-TreeSHAP: Depth-Independent TreeSHAP and Shapley Interactions](https://mlsanity.com/paper/2605.04497/) (2026)
- [DELATOR: Money Laundering Detection via Multi-Task Learning on Large Transaction Graphs](https://mlsanity.com/paper/2205.10293/) (2022)
- [Machine Learning Methods Applied to Cortico-Cortical Evoked Potentials Aid in Localizing Seizure Onset Zones](https://mlsanity.com/paper/2211.07867/) (2022)
- [Heterogeneous Random Forest](https://mlsanity.com/paper/2410.19022/) (2024)
- [Graph-Linguistic Fusion: Using Language Models for Wikidata Vandalism Detection](https://mlsanity.com/paper/2505.18136/) (2025)
- [Tabular foundation models for robust calibration of near-infrared chemical sensing data](https://mlsanity.com/paper/2605.21544/) (2026)
- [Generalized Linear Tree Space Nearest Neighbor](https://mlsanity.com/paper/2103.16408/) (2021)
- [Machine Learning for Real-World Evidence Analysis of COVID-19 Pharmacotherapy](https://mlsanity.com/paper/2107.10239/) (2021)
- [A multiscale spatiotemporal approach for smallholder irrigation detection](https://mlsanity.com/paper/2202.04239/) (2022)
- [Optimization of Oblivious Decision Tree Ensembles Evaluation for CPU](https://mlsanity.com/paper/2211.00391/) (2022)
- [Performance and utility trade-off in interpretable sleep staging](https://mlsanity.com/paper/2211.03282/) (2022)
- [A generalized decision tree ensemble based on the NeuralNetworks architecture: Distributed Gradient Boosting Forest (DGBF)](https://mlsanity.com/paper/2402.03386/) (2024)
- [Breast Cancer Classification Using Gradient Boosting Algorithms Focusing on Reducing the False Negative and SHAP for Explainability](https://mlsanity.com/paper/2403.09548/) (2024)
- [Explainable convolutional neural network model provides an alternative genome-wide association perspective on mutations in SARS-CoV-2](https://mlsanity.com/paper/2410.22452/) (2024)
- [Predicting Emergency Department Visits for Patients with Type II Diabetes](https://mlsanity.com/paper/2412.08984/) (2024)
- [Comparative Analysis of Modern Machine Learning Models for Retail Sales Forecasting](https://mlsanity.com/paper/2506.05941/) (2025)
- [WOODELF-HD: Efficient Background SHAP for High-Depth Decision Trees](https://mlsanity.com/paper/2604.10569/) (2026)
- [Adversarial Creation and Detection of AI-Generated Social Bot Content](https://mlsanity.com/paper/2606.07219/) (2026)
- [Uncertainty quantification in mechanics: A unified Bayesian perspective](https://mlsanity.com/paper/2607.18734/) (2026)
- [Scaling Laws, Tabular Data and Actuarial Ratemaking Models](https://mlsanity.com/paper/2609.03106/) (2026)
- [Which alloy composition,what process parameters? Inferring the recipe from optimized metallic microstructure and texture](https://mlsanity.com/paper/2610.08165/) (2026)

## Abstract (arXiv)

This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms. In this paper, we provide a detailed analysis of this problem and demonstrate that proposed algorithms solve it effectively, leading to excellent empirical results.