mlsanityData to 14 Oct 2026

SIGMA: SHAP-Guided Implicit-Trajectory Generation for Metadata-Free LLM-Based AutoFE

arXiv 2608.17948 ↗First posted 18 Aug 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent research has leveraged Large Language Models (LLMs) to enhance Automated Feature Engineering (AutoFE) through semantic descriptions and trajectory-based prompting. However, there exist two challenges that limit their applicability and scalability in long-horizon optimization: (1) semantic metadata is unavailable in many practical settings, and (2) trajectory accumulation increases the risk of exceeding the context window, while without it, the generation process can become unstable, leading to becoming stuck in the local optima and a high duplicate rate of generated features. To this end, we propose a SHAP-enhanced Implicit-trajectory Generation for Metadata-free AutoFE (SIGMA), a scalable constant-context optimization framework. SIGMA leverages SHAP values to provide task-aware signals for guiding group feature generation instead of semantic information. In addition, we adopt an EXposed-feature Implicit Trajectory (EXIT) approach, where the exposed features in the prompt implicitly represent the trajectory. Empirical results demonstrate that SIGMA achieves performance comparable to the state-of-the-art (SOTA) LLM baselines with a nearly constant prompt length. Notably, EXIT significantly reduces the duplicate ratio of generated features from 37.2% to 6.8%. At the same time, SIGMA matches traditional SOTA performance with only 5.4 features on average, demonstrating substantial efficiency gains in feature utilization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

20 of 33 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  7. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  8. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 127
  9. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 125
  10. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 81
  11. 2019The autofeat Python Library for Automated Feature Engineering and Selectioncited by 73
  12. 2019OpenML-Python: an extensible Python API for OpenMLcited by 63
8 more
  1. 2022OpenFE: Automated Feature Generation with Expert-level Performancecited by 52
  2. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoningcited by 49
  3. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizerscited by 38
  4. 2025Large Language Model-based Data Science Agent: A Surveycited by 23
  5. 2024ELF-Gym: Evaluating Large Language Models Generated Features for Tabular Predictioncited by 10
  6. 2024Large Language Models Engineer Too Many Simple Features For Tabular Datacited by 10
  7. 2019Towards automated feature engineering for credit card fraud detection using multi-perspective HMMscited by 7
  8. 2025Overtuning in Hyperparameter Optimizationcited by 5

Built on by

Not cited yet.