# Combining Deep Learning with Good Old-Fashioned Machine Learning

arXiv: https://arxiv.org/abs/2207.03757 · HTML: https://mlsanity.com/paper/2207.03757/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 29 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 5 of 5)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [Neural Networks with A La Carte Selection of Activation Functions](https://mlsanity.com/paper/2206.12166/) (2022)

## Built on by (most cited first, 2 of 2)

- [Classy Ensemble: A Novel Ensemble Algorithm for Classification](https://mlsanity.com/paper/2302.10580/) (2023)
- [A Melting Pot of Evolution and Learning](https://mlsanity.com/paper/2306.04971/) (2023)

## Abstract (arXiv)

We present a comprehensive, stacking-based framework for combining deep learning with good old-fashioned machine learning, called Deep GOld. Our framework involves ensemble selection from 51 retrained pretrained deep networks as first-level models, and 10 machine-learning algorithms as second-level models. Enabled by today's state-of-the-art software tools and hardware platforms, Deep GOld delivers consistent improvement when tested on four image-classification datasets: Fashion MNIST, CIFAR10, CIFAR100, and Tiny ImageNet. Of 120 experiments, in all but 10 Deep GOld improved the original networks' performance.