# FAMOSE: A ReAct Approach to Automated Feature Discovery

arXiv: https://arxiv.org/abs/2602.17641 · HTML: https://mlsanity.com/paper/2602.17641/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 58 references that are in the map.
- First posted on arXiv 19 Feb 2026.

## Builds on (most cited first, 22 of 22)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data](https://mlsanity.com/paper/2003.06505/) (2020)
- [Large Language Models as General Pattern Machines](https://mlsanity.com/paper/2307.04721/) (2023)
- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks](https://mlsanity.com/paper/2206.06565/) (2022)
- [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://mlsanity.com/paper/2510.22954/) (2025)
- [Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineering](https://mlsanity.com/paper/2305.03403/) (2023)
- [Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning](https://mlsanity.com/paper/2404.09491/) (2024)
- [The autofeat Python Library for Automated Feature Engineering and Selection](https://mlsanity.com/paper/1901.07329/) (2019)
- [OpenFE: Automated Feature Generation with Expert-level Performance](https://mlsanity.com/paper/2211.12507/) (2022)
- [Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning](https://mlsanity.com/paper/2406.08527/) (2024)
- [Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation](https://mlsanity.com/paper/2211.13358/) (2022)
- [LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers](https://mlsanity.com/paper/2503.14434/) (2025)
- [Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models](https://mlsanity.com/paper/2404.06209/) (2024)
- [An Empirical Analysis of Feature Engineering for Predictive Modeling](https://mlsanity.com/paper/1701.07852/) (2017)
- [A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data](https://mlsanity.com/paper/2407.02112/) (2024)
- [An LLM-Based Approach for Insight Generation in Data Analysis](https://mlsanity.com/paper/2503.11664/) (2025)
- [Endgame Analysis of Dou Shou Qi](https://mlsanity.com/paper/1604.07312/) (2016)
- [ELF-Gym: Evaluating Large Language Models Generated Features for Tabular Prediction](https://mlsanity.com/paper/2410.12865/) (2024)
- [UniPredict: Large Language Models are Universal Tabular Classifiers](https://mlsanity.com/paper/2310.03266/) (2023)
- [Data Analysis in the Era of Generative AI](https://mlsanity.com/paper/2409.18475/) (2024)
- [FastFT: Accelerating Reinforced Feature Transformation via Advanced Exploration Strategies](https://mlsanity.com/paper/2503.20394/) (2025)

## Built on by (most cited first, 4 of 4)

- [FeatEHR-LLM: Leveraging Large Language Models for Feature Engineering in Electronic Health Records](https://mlsanity.com/paper/2604.22534/) (2026)
- [Adaptive Graph-of-Islands Evolution for Automatic Feature Engineering with LLMs](https://mlsanity.com/paper/2607.23286/) (2026)
- [The Geometry of Logic: Stratification Induces Semantic Structure and Robust Reasoning](https://mlsanity.com/paper/2609.32927/) (2026)
- [Long Text to Predictive Features: LLM-Guided Blockwise Feature Engineering via Executable Program Search](https://mlsanity.com/paper/2610.12390/) (2026)

## Abstract (arXiv)

Feature engineering remains a critical yet challenging bottleneck in machine learning, particularly for tabular data, as identifying optimal features from an exponentially large feature space traditionally demands substantial domain expertise. To address this challenge, we introduce FAMOSE (Feature AugMentation and Optimal Selection agEnt), a novel framework that leverages the ReAct paradigm to autonomously explore, generate, and refine features while integrating feature selection and evaluation tools within an agent architecture. To our knowledge, FAMOSE represents the first application of an agentic ReAct framework to automated feature engineering, especially for both regression and classification tasks. Extensive experiments demonstrate that FAMOSE is at or near the state-of-the-art on classification tasks (especially tasks with more than 10K instances, where ROC-AUC increases 0.23% on average), and achieves the state-of-the-art for regression tasks by reducing RMSE by 2.0% on average, while remaining more robust to errors than other algorithms. We hypothesize that FAMOSE's strong performance is because ReAct allows the LLM context window to record (via iterative feature discovery and evaluation steps) what features did or did not work. This is similar to a few-shot prompt and guides the LLM to invent better, more innovative features. Our work offers evidence that AI agents are remarkably effective in solving problems that require highly inventive solutions, such as feature engineering.