mlsanityData to 7 Oct 2026

ICE-SEARCH: A Language Model-Driven Feature Selection Approach

arXiv 2402.18609 ↗First posted 28 Feb 2024cs.LG cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

This study unveils the In-Context Evolutionary Search (ICE-SEARCH) method, which is among the first works that melds large language models (LLMs) with evolutionary algorithms for feature selection (FS) tasks and demonstrates its effectiveness in Medical Predictive Analytics (MPA) applications. ICE-SEARCH harnesses the crossover and mutation capabilities inherent in LLMs within an evolutionary framework, significantly improving FS through the model's comprehensive world knowledge and its adaptability to a variety of roles. Our evaluation of this methodology spans three crucial MPA tasks: stroke, cardiovascular disease, and diabetes, where ICE-SEARCH outperforms traditional FS methods in pinpointing essential features for medical applications. ICE-SEARCH achieves State-of-the-Art (SOTA) performance in stroke prediction and diabetes prediction; the Decision-Randomized ICE-SEARCH ranks as SOTA in cardiovascular disease prediction. The study emphasizes the critical role of incorporating domain-specific insights, illustrating ICE-SEARCH's robustness, generalizability, and convergence. This opens avenues for further research into comprehensive and intricate FS landscapes, marking a significant stride in the application of artificial intelligence in medical predictive analytics.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 30 of its 70 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

30 of 70 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  9. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  12. 2023BloombergGPT: A Large Language Model for Financecited by 923
18 more
  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  2. 2023Large Language Models as Optimizerscited by 830
  3. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  4. 2021MetaICL: Learning to Learn In Contextcited by 450
  5. 2022Language Models are Realistic Tabular Data Generatorscited by 290
  6. 2023Role-Play with Large Language Modelscited by 290
  7. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 220
  8. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  9. 2020The Computational Limits of Deep Learningcited by 188
  10. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  11. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  12. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 125
  13. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 119
  14. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  15. 2020Learning from Task Descriptionscited by 78
  16. 2015Survey on Feature Selectioncited by 25
  17. 2023Optimizing Feature Set for Click-Through Rate Predictioncited by 13
  18. 2023Empowering Multi-step Reasoning across Languages via Tree-of-Thoughtscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 3.

8 papers.

  1. 2024Exploring Large Language Models for Feature Selection: A Data-centric Perspectivecited by 27
  2. 2025LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularizationcited by 10
  3. 2024Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Modelscited by 9
  4. 2025LLM4FS: Leveraging Large Language Models for Feature Selectioncited by 2
  5. 2026LLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systemscited by 1
  6. 2025LLM-FS-Agent: A Deliberative Role-based Large Language Model Architecture for Transparent Feature Selectioncited by 0
  7. 2026LLM Sparsity Prior for Robust Feature Selectioncited by 0
  8. 2026LLM-Enhanced Multi-Agent Reinforcement Learning for Unified Electric Vehicles-Charging Station-Grid Optimization in Public Charging Systemscited by 0