# Toward Efficient Automated Feature Engineering

arXiv: https://arxiv.org/abs/2212.13152 · HTML: https://mlsanity.com/paper/2212.13152/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 26 Dec 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Feature Engineering for Predictive Modeling using Reinforcement Learning](https://mlsanity.com/paper/1709.07150/) (2017)
- [Group-wise Reinforcement Feature Generation for Optimal and Explainable Representation Space Reconstruction](https://mlsanity.com/paper/2205.14526/) (2022)
- [AutoFS: Automated Feature Selection via Diversity-aware Interactive Reinforcement Learning](https://mlsanity.com/paper/2008.12001/) (2020)
- [Efficient Reinforced Feature Selection via Early Stopping Traverse Strategy](https://mlsanity.com/paper/2109.14180/) (2021)
- [Interactive Reinforcement Learning for Feature Selection with Decision Tree in the Loop](https://mlsanity.com/paper/2010.02506/) (2020)
- [SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks](https://mlsanity.com/paper/2003.02556/) (2020)
- [Simplifying Reinforced Feature Selection via Restructured Choice Strategy of Single Agent](https://mlsanity.com/paper/2009.09230/) (2020)
- [Characterizing classification datasets: a study of meta-features for meta-learning](https://mlsanity.com/paper/1808.10406/) (2018)
- [A Review for Weighted MinHash Algorithms](https://mlsanity.com/paper/1811.04633/) (2018)

## Built on by (most cited first, 2 of 2)

- [SparseDM: Toward Sparse Efficient Diffusion Models](https://mlsanity.com/paper/2404.10445/) (2024)
- [Dataset-Agnostic Recommender Systems](https://mlsanity.com/paper/2501.07294/) (2025)

## Abstract (arXiv)

Automated Feature Engineering (AFE) refers to automatically generate and select optimal feature sets for downstream tasks, which has achieved great success in real-world applications. Current AFE methods mainly focus on improving the effectiveness of the produced features, but ignoring the low-efficiency issue for large-scale deployment. Therefore, in this work, we propose a generic framework to improve the efficiency of AFE. Specifically, we construct the AFE pipeline based on reinforcement learning setting, where each feature is assigned an agent to perform feature transformation comand selection, and the evaluation score of the produced features in downstream tasks serve as the reward to update the policy. We improve the efficiency of AFE in two perspectives. On the one hand, we develop a Feature Pre-Evaluation (FPE) Model to reduce the sample size and feature size that are two main factors on undermining the efficiency of feature evaluation. On the other hand, we devise a two-stage policy training strategy by running FPE on the pre-evaluation task as the initialization of the policy to avoid training policy from scratch. We conduct comprehensive experiments on 36 datasets in terms of both classification and regression tasks. The results show 2.9% higher performance in average and 2x higher computational efficiency comparing to state-of-the-art AFE methods.