mlsanityData to 14 Oct 2026

Feature Generation Using LLMs: An Evolutionary Algorithm Approach

arXiv 2607.16255 ↗First posted 27 Jun 2026cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

A crucial step in machine learning pipelines is to present each entity with features or attributes that are representative of the characteristics of the processed entities. Feature engineering is an important step in finding a relation among attributes that otherwise may not be processed by the ML algorithms. Meanwhile, Large Language Models have shown promising abilities in coding, mathematical reasoning, and processing world knowledge. In this work, we utilize an LLM for the problem of feature generation from tabular data based on the previously given features. We have created a pipeline that takes a set of attributes and a prompt to generate new features. Then, our selection algorithm selects the best-performing sets of attributes. We apply our method to eight datasets from different domains and data types. Our results show that, in most cases, the language model can produce new features based on mathematical and logical operators that are useful for the given tasks and can improve classification results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 39 references that are in the map. First posted on arXiv 27 Jun 2026.

Builds on

13 of 39 references.

References · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  3. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  4. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 284
  5. 2023Large Language Models in Finance: A Surveycited by 226
  6. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 122
  7. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 90
  8. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 81
  9. 2017Feature Engineering for Predictive Modeling using Reinforcement Learningcited by 78
  10. 2024LLM-Select: Feature Selection with Large Language Modelscited by 30
  11. 2017An Empirical Analysis of Feature Engineering for Predictive Modelingcited by 29
  12. 2024Dynamic and Adaptive Feature Generation with LLMcited by 27
1 more
  1. 2024Exploring Large Language Models for Feature Selection: A Data-centric Perspectivecited by 27

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers
  2. 2025Tabular Feature Discovery With Reasoning Type Exploration
  3. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning
  4. 2025FeRG-LLM : Feature Engineering by Reason Generation Large Language Models
  5. 2025Applications of Large Language Model Reasoning in Feature Generation
  6. 2026Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data
  7. 2025A Survey on Data-Centric AI: Tabular Learning from Reinforcement Learning and Generative AI Perspective
  8. 2025TAGAL: Tabular Data Generation using Agentic LLM Methods