# Embedding and Extraction of Knowledge in Tree Ensemble Classifiers

arXiv: https://arxiv.org/abs/2010.08281 · HTML: https://mlsanity.com/paper/2010.08281/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Survey](https://mlsanity.com/paper/2409.18214/) (2024)
- [MAIDS: Malicious Agent Identification-based Data Security Model for Cloud Environments](https://mlsanity.com/paper/2412.14490/) (2024)
- [Backdoor attacks on DNN and GBDT -- A Case Study from the insurance domain](https://mlsanity.com/paper/2412.08366/) (2024)
- [Context-Aware Scientific Knowledge Extraction on Linked Open Data using Large Language Models](https://mlsanity.com/paper/2506.17580/) (2025)
- [Leveraging System-Level Observations to Inform Bayesian Learning of Model Parameters for Quantitative Verification](https://mlsanity.com/paper/2608.03489/) (2026)

## Abstract (arXiv)

The embedding and extraction of useful knowledge is a recent trend in machine learning applications, e.g., to supplement existing datasets that are small. Whilst, as the increasing use of machine learning models in security-critical applications, the embedding and extraction of malicious knowledge are equivalent to the notorious backdoor attack and its defence, respectively. This paper studies the embedding and extraction of knowledge in tree ensemble classifiers, and focuses on knowledge expressible with a generic form of Boolean formulas, e.g., robustness properties and backdoor attacks. For the embedding, it is required to be preservative(the original performance of the classifier is preserved), verifiable(the knowledge can be attested), and stealthy(the embedding cannot be easily detected). To facilitate this, we propose two novel, and effective, embedding algorithms, one of which is for black-box settings and the other for white-box settings.The embedding can be done in PTIME. Beyond the embedding, we develop an algorithm to extract the embedded knowledge, by reducing the problem to be solvable with an SMT (satisfiability modulo theories) solver. While this novel algorithm can successfully extract knowledge, the reduction leads to an NP computation. Therefore, if applying embedding as backdoor attacks and extraction as defence, our results suggest a complexity gap (P vs. NP) between the attack and defence when working with tree ensemble classifiers. We apply our algorithms toa diverse set of datasets to validate our conclusion extensively.