mlsanityData to 7 Oct 2026

FAMOSE: A ReAct Approach to Automated Feature Discovery

arXiv 2602.17641 ↗First posted 19 Feb 2026cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Feature engineering remains a critical yet challenging bottleneck in machine learning, particularly for tabular data, as identifying optimal features from an exponentially large feature space traditionally demands substantial domain expertise. To address this challenge, we introduce FAMOSE (Feature AugMentation and Optimal Selection agEnt), a novel framework that leverages the ReAct paradigm to autonomously explore, generate, and refine features while integrating feature selection and evaluation tools within an agent architecture. To our knowledge, FAMOSE represents the first application of an agentic ReAct framework to automated feature engineering, especially for both regression and classification tasks. Extensive experiments demonstrate that FAMOSE is at or near the state-of-the-art on classification tasks (especially tasks with more than 10K instances, where ROC-AUC increases 0.23% on average), and achieves the state-of-the-art for regression tasks by reducing RMSE by 2.0% on average, while remaining more robust to errors than other algorithms. We hypothesize that FAMOSE's strong performance is because ReAct allows the LLM context window to record (via iterative feature discovery and evaluation steps) what features did or did not work. This is similar to a few-shot prompt and guides the LLM to invent better, more innovative features. Our work offers evidence that AI agents are remarkably effective in solving problems that require highly inventive solutions, such as feature engineering.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 22 of its 58 references that are in the map. First posted on arXiv 19 Feb 2026.

Builds on

22 of 58 references.

References · 22

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 368
  3. 2023Large Language Models as General Pattern Machinescited by 196
  4. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  5. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  6. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
  7. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 125
  8. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 79
  9. 2019The autofeat Python Library for Automated Feature Engineering and Selectioncited by 73
  10. 2022OpenFE: Automated Feature Generation with Expert-level Performancecited by 52
  11. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoningcited by 48
  12. 2022Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluationcited by 39
10 more
  1. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizerscited by 35
  2. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  3. 2017An Empirical Analysis of Feature Engineering for Predictive Modelingcited by 29
  4. 2024A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Datacited by 15
  5. 2025An LLM-Based Approach for Insight Generation in Data Analysiscited by 12
  6. 2016Endgame Analysis of Dou Shou Qicited by 11
  7. 2024ELF-Gym: Evaluating Large Language Models Generated Features for Tabular Predictioncited by 10
  8. 2023UniPredict: Large Language Models are Universal Tabular Classifierscited by 9
  9. 2024Data Analysis in the Era of Generative AIcited by 6
  10. 2025FastFT: Accelerating Reinforced Feature Transformation via Advanced Exploration Strategiescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026FeatEHR-LLM: Leveraging Large Language Models for Feature Engineering in Electronic Health Recordscited by 1
  2. 2026Adaptive Graph-of-Islands Evolution for Automatic Feature Engineering with LLMscited by 0
  3. 2026The Geometry of Logic: Stratification Induces Semantic Structure and Robust Reasoningcited by 0
  4. 2026Long Text to Predictive Features: LLM-Guided Blockwise Feature Engineering via Executable Program Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AutoData: Agentic Search for Pre-training Data Selection
  2. 2025FeRG-LLM : Feature Engineering by Reason Generation Large Language Models
  3. 2026CoFEE: Reasoning Control for LLM-Based Feature Discovery
  4. 2026CoFEH: LLM-driven Feature Engineering Empowered by Collaborative Bayesian Hyperparameter Optimization
  5. 2026TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks
  6. 2026Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data
  7. 2026Synthesizing Feature Extractors: An Agentic Approach for Algorithm Selection
  8. 2026Automatic Prompt Optimization for Dataset-Level Feature Discovery