# OpenFE: Automated Feature Generation with Expert-level Performance

arXiv: https://arxiv.org/abs/2211.12507 · HTML: https://mlsanity.com/paper/2211.12507/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 56 references that are in the map.
- First posted on arXiv 22 Nov 2022.

## Builds on (most cited first, 17 of 17)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data](https://mlsanity.com/paper/1909.06312/) (2019)
- [On Embeddings for Numerical Features in Tabular Deep Learning](https://mlsanity.com/paper/2203.05556/) (2022)
- [OpenML Benchmarking Suites](https://mlsanity.com/paper/1708.03731/) (2017)
- [Introducing LETOR 4.0 Datasets](https://mlsanity.com/paper/1306.2597/) (2013)
- [Feature Engineering for Predictive Modeling using Reinforcement Learning](https://mlsanity.com/paper/1709.07150/) (2017)
- [The autofeat Python Library for Automated Feature Engineering and Selection](https://mlsanity.com/paper/1901.07329/) (2019)
- [AutoCross: Automatic Feature Crossing for Tabular Data in Real-World Applications](https://mlsanity.com/paper/1904.12857/) (2019)
- [SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks](https://mlsanity.com/paper/2003.02556/) (2020)
- [DIFER: Differentiable Automated Feature Engineering](https://mlsanity.com/paper/2010.08784/) (2020)

## Built on by (most cited first, 30 of 52)

- [MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement](https://mlsanity.com/paper/2506.15692/) (2025)
- [Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning](https://mlsanity.com/paper/2406.08527/) (2024)
- [LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers](https://mlsanity.com/paper/2503.14434/) (2025)
- [Navigating the Alpha Jungle: An LLM-Powered MCTS Framework for Formulaic Factor Mining](https://mlsanity.com/paper/2505.11122/) (2025)
- [Cognitive Alpha Mining via LLM-Driven Code-Based Evolution](https://mlsanity.com/paper/2511.18850/) (2025)
- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [QuantFactor REINFORCE: Mining Steady Formulaic Alpha Factors with Variance-bounded REINFORCE](https://mlsanity.com/paper/2409.05144/) (2024)
- [Unsupervised Feature Transformation via In-context Generation, Generator-critic LLM Agents, and Duet-play Teaming](https://mlsanity.com/paper/2504.21304/) (2025)
- [Large Language Models Engineer Too Many Simple Features For Tabular Data](https://mlsanity.com/paper/2410.17787/) (2024)
- [LLM Meeting Decision Trees on Tabular Data](https://mlsanity.com/paper/2505.17918/) (2025)
- [Autonomous Data Agents: A New Opportunity for Smart Data](https://mlsanity.com/paper/2509.18710/) (2025)
- [Enhancing Tabular Data Optimization with a Flexible Graph-based Reinforced Exploration Strategy](https://mlsanity.com/paper/2406.07404/) (2024)
- [The FIX Benchmark: Extracting Features Interpretable to eXperts](https://mlsanity.com/paper/2409.13684/) (2024)
- [FastFT: Accelerating Reinforced Feature Transformation via Advanced Exploration Strategies](https://mlsanity.com/paper/2503.20394/) (2025)
- [LLM-ML Teaming: Integrated Symbolic Decoding and Gradient Search for Valid and Stable Generative Feature Transformation](https://mlsanity.com/paper/2506.09085/) (2025)
- [Learning from Expert Factors: Trajectory-level Reward Shaping for Formulaic Alpha Mining](https://mlsanity.com/paper/2507.20263/) (2025)
- [IIFE: Interaction Information Based Automated Feature Engineering](https://mlsanity.com/paper/2409.04665/) (2024)
- [FAMOSE: A ReAct Approach to Automated Feature Discovery](https://mlsanity.com/paper/2602.17641/) (2026)
- [Retrieval-Augmented Feature Generation for Domain-Specific Classification](https://mlsanity.com/paper/2406.11177/) (2024)
- [Tabular Feature Discovery With Reasoning Type Exploration](https://mlsanity.com/paper/2506.20357/) (2025)
- [Collaborative Multi-Agent Reinforcement Learning for Automated Feature Transformation with Graph-Driven Path Optimization](https://mlsanity.com/paper/2504.17355/) (2025)
- [FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Data](https://mlsanity.com/paper/2510.25223/) (2025)
- [Knowledge-Informed Automatic Feature Extraction via Collaborative Large Language Model Agents](https://mlsanity.com/paper/2511.15074/) (2025)
- [AlphaForgeBench: Benchmarking End-to-End Trading Strategy Design with Large Language Models](https://mlsanity.com/paper/2602.18481/) (2026)
- [SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering](https://mlsanity.com/paper/2607.04163/) (2026)
- [MDArena: Evaluating Coding Agents on Realistic Molecular Dynamics Workflows](https://mlsanity.com/paper/2608.02642/) (2026)
- [Federated Automated Feature Engineering](https://mlsanity.com/paper/2412.04404/) (2024)
- [Dataset-Agnostic Recommender Systems](https://mlsanity.com/paper/2501.07294/) (2025)
- [How Usable is Automated Feature Engineering for Tabular Data?](https://mlsanity.com/paper/2508.13932/) (2025)
- [Dataforge: Agentic Platform for Autonomous Data Engineering](https://mlsanity.com/paper/2511.06185/) (2025)

## Abstract (arXiv)

The goal of automated feature generation is to liberate machine learning experts from the laborious task of manual feature generation, which is crucial for improving the learning performance of tabular data. The major challenge in automated feature generation is to efficiently and accurately identify effective features from a vast pool of candidate features. In this paper, we present OpenFE, an automated feature generation tool that provides competitive results against machine learning experts. OpenFE achieves high efficiency and accuracy with two components: 1) a novel feature boosting method for accurately evaluating the incremental performance of candidate features and 2) a two-stage pruning algorithm that performs feature pruning in a coarse-to-fine manner. Extensive experiments on ten benchmark datasets show that OpenFE outperforms existing baseline methods by a large margin. We further evaluate OpenFE in two Kaggle competitions with thousands of data science teams participating. In the two competitions, features generated by OpenFE with a simple baseline model can beat 99.3% and 99.6% data science teams respectively. In addition to the empirical results, we provide a theoretical perspective to show that feature generation can be beneficial in a simple yet representative setting. The code is available at https://github.com/ZhangTP1996/OpenFE.