# DIFER: Differentiable Automated Feature Engineering

arXiv: https://arxiv.org/abs/2010.08784 · HTML: https://mlsanity.com/paper/2010.08784/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2020.

## Built on by (most cited first, 16 of 16)

- [OpenFE: Automated Feature Generation with Expert-level Performance](https://mlsanity.com/paper/2211.12507/) (2022)
- [Reinforcement-Enhanced Autoregressive Feature Transformation: Gradient-steered Search in Continuous Space for Postfix Expressions](https://mlsanity.com/paper/2309.13618/) (2023)
- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [Enhancing Tabular Data Optimization with a Flexible Graph-based Reinforced Exploration Strategy](https://mlsanity.com/paper/2406.07404/) (2024)
- [FastFT: Accelerating Reinforced Feature Transformation via Advanced Exploration Strategies](https://mlsanity.com/paper/2503.20394/) (2025)
- [IIFE: Interaction Information Based Automated Feature Engineering](https://mlsanity.com/paper/2409.04665/) (2024)
- [ELATE: Evolutionary Language model for Automated Time-series Engineering](https://mlsanity.com/paper/2508.14667/) (2025)
- [Collaborative Multi-Agent Reinforcement Learning for Automated Feature Transformation with Graph-Driven Path Optimization](https://mlsanity.com/paper/2504.17355/) (2025)
- [RuDi: Explaining Behavior Sequence Models by Automatic Statistics Generation and Rule Distillation](https://mlsanity.com/paper/2208.07211/) (2022)
- [Federated Automated Feature Engineering](https://mlsanity.com/paper/2412.04404/) (2024)
- [How Usable is Automated Feature Engineering for Tabular Data?](https://mlsanity.com/paper/2508.13932/) (2025)
- [GPT-FT: An Efficient Automated Feature Transformation Using GPT for Sequence Reconstruction and Performance Enhancement](https://mlsanity.com/paper/2508.20824/) (2025)
- [FeatGeNN: Improving Model Performance for Tabular Data with Correlation-based Feature Extraction](https://mlsanity.com/paper/2308.07527/) (2023)
- [Two-Stage Feature Generation with Transformer and Reinforcement Learning](https://mlsanity.com/paper/2505.21978/) (2025)
- [Universal Reusability in Recommender Systems: The Case for Dataset- and Task-Independent Frameworks](https://mlsanity.com/paper/2506.03391/) (2025)
- [LATTEArena: An Evaluation Framework for LLM-powered Tabular Feature Engineering (Extended Version)](https://mlsanity.com/paper/2606.09004/) (2026)

## Abstract (arXiv)

Feature engineering, a crucial step of machine learning, aims to extract useful features from raw data to improve data quality. In recent years, great efforts have been devoted to Automated Feature Engineering (AutoFE) to replace expensive human labor. However, existing methods are computationally demanding due to treating AutoFE as a coarse-grained black-box optimization problem over a discrete space. In this work, we propose an efficient gradient-based method called DIFER to perform differentiable automated feature engineering in a continuous vector space. DIFER selects potential features based on evolutionary algorithm and leverages an encoder-predictor-decoder controller to optimize existing features. We map features into the continuous vector space via the encoder, optimize the embedding along the gradient direction induced by the predicted score, and recover better features from the optimized embedding by the decoder. Extensive experiments on classification and regression datasets demonstrate that DIFER can significantly improve the performance of various machine learning algorithms and outperform current state-of-the-art AutoFE methods in terms of both efficiency and performance.