# An Empirical Analysis of Feature Engineering for Predictive Modeling

arXiv: https://arxiv.org/abs/1701.07852 · HTML: https://mlsanity.com/paper/1701.07852/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2017.

## Built on by (most cited first, 29 of 29)

- [RelBench: A Benchmark for Deep Learning on Relational Databases](https://mlsanity.com/paper/2407.20060/) (2024)
- [ERASE: Benchmarking Feature Selection Methods for Deep Recommender Systems](https://mlsanity.com/paper/2403.12660/) (2024)
- [FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches](https://mlsanity.com/paper/2103.14539/) (2021)
- [RAIDER: Reinforcement-aided Spear Phishing Detector](https://mlsanity.com/paper/2105.07582/) (2021)
- [Symbolic Regression as Feature Engineering Method for Machine and Deep Learning Regression Tasks](https://mlsanity.com/paper/2311.06028/) (2023)
- [FAMOSE: A ReAct Approach to Automated Feature Discovery](https://mlsanity.com/paper/2602.17641/) (2026)
- [Pairing Conceptual Modeling with Machine Learning](https://mlsanity.com/paper/2106.14251/) (2021)
- [TREE-G: Decision Trees Contesting Graph Neural Networks](https://mlsanity.com/paper/2207.02760/) (2022)
- [Energy reconstruction for large liquid scintillator detectors with machine learning techniques: aggregated features approach](https://mlsanity.com/paper/2206.09040/) (2022)
- [MACFE: A Meta-learning and Causality Based Feature Engineering Framework](https://mlsanity.com/paper/2207.04010/) (2022)
- [Classification of integers based on residue classes via modern deep learning algorithms](https://mlsanity.com/paper/2304.01333/) (2023)
- [Feature Engineering for Scalable Application-Level Post-Silicon Debugging](https://mlsanity.com/paper/2102.04554/) (2021)
- [DRAM Failure Prediction in AIOps: Empirical Evaluation, Challenges and Opportunities](https://mlsanity.com/paper/2104.15052/) (2021)
- [Res-CNN-BiLSTM Network for overcoming Mental Health Disturbances caused due to Cyberbullying through Social Media](https://mlsanity.com/paper/2204.09738/) (2022)
- [GCN-WP -- Semi-Supervised Graph Convolutional Networks for Win Prediction in Esports](https://mlsanity.com/paper/2207.13191/) (2022)
- [Introducing 'Inside' Out of Distribution](https://mlsanity.com/paper/2407.04534/) (2024)
- [RCCFormer: A Robust Crowd Counting Network Based on Transformer](https://mlsanity.com/paper/2504.04935/) (2025)
- [Assessing the Effectiveness of Deep Embeddings for Tree Species Classification in the Dutch Forest Inventory](https://mlsanity.com/paper/2508.18829/) (2025)
- [Machine Learning Framework: Competitive Intelligence and Key Drivers Identification of Market Share Trends Among Healthcare Facilities](https://mlsanity.com/paper/2212.04810/) (2022)
- [Classification of social media Toxic comments using Machine learning models](https://mlsanity.com/paper/2304.06934/) (2023)
- [Model Explainability in Physiological and Healthcare-based Neural Networks](https://mlsanity.com/paper/2304.14495/) (2023)
- [Riemann-Lebesgue Forest for Regression](https://mlsanity.com/paper/2402.04550/) (2024)
- [AI-Powered Prediction of Nanoparticle Pharmacokinetics: A Multi-View Learning Approach](https://mlsanity.com/paper/2503.13798/) (2025)
- [Predicting and Explaining Customer Data Sharing in the Open Banking](https://mlsanity.com/paper/2507.01987/) (2025)
- [Deep Learning Multi-Horizon Irradiance Nowcasting: A Comparative Evaluation of Three Methods for Leveraging Sky Images](https://mlsanity.com/paper/2603.26704/) (2026)
- [CoFEE: Reasoning Control for LLM-Based Feature Discovery](https://mlsanity.com/paper/2604.21584/) (2026)
- [Feature Generation Using LLMs: An Evolutionary Algorithm Approach](https://mlsanity.com/paper/2607.16255/) (2026)
- [The impact of feature engineering and an optimisation framework for ocean colour machine learning](https://mlsanity.com/paper/2608.19899/) (2026)
- [Do Tabular Foundation Models Still Need Feature Engineering?](https://mlsanity.com/paper/2609.13202/) (2026)

## Abstract (arXiv)

Machine learning models, such as neural networks, decision trees, random forests, and gradient boosting machines, accept a feature vector, and provide a prediction. These models learn in a supervised fashion where we provide feature vectors mapped to the expected output. It is common practice to engineer new features from the provided feature set. Such engineered features will either augment or replace portions of the existing feature vector. These engineered features are essentially calculated fields based on the values of the other features. Engineering such features is primarily a manual, time-consuming task. Additionally, each type of model will respond differently to different kinds of engineered features. This paper reports empirical research to demonstrate what kinds of engineered features are best suited to various machine learning model types. We provide this recommendation by generating several datasets that we designed to benefit from a particular type of engineered feature. The experiment demonstrates to what degree the machine learning model can synthesize the needed feature on its own. If a model can synthesize a planned feature, it is not necessary to provide that feature. The research demonstrated that the studied models do indeed perform differently with various types of engineered features.