mlsanityData to 14 Oct 2026

Exploring Large Language Models for Feature Selection: A Data-centric Perspective

arXiv 2408.12025 ↗First posted 21 Aug 2024cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

The rapid advancement of Large Language Models (LLMs) has significantly influenced various domains, leveraging their exceptional few-shot and zero-shot learning capabilities. In this work, we aim to explore and understand the LLMs-based feature selection methods from a data-centric perspective. We begin by categorizing existing feature selection methods with LLMs into two groups: data-driven feature selection which requires numerical values of samples to do statistical inference and text-based feature selection which utilizes prior knowledge of LLMs to do semantical associations using descriptive context. We conduct experiments in both classification and regression tasks with LLMs in various sizes (e.g., GPT-4, ChatGPT and LLaMA-2). Our findings emphasize the effectiveness and robustness of text-based feature selection methods and showcase their potentials using a real-world medical application. We also discuss the challenges and future opportunities in employing LLMs for feature selection, offering insights for further research and development in this emerging field.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 43 of its 75 references that are in the map. First posted on arXiv 21 Aug 2024.

Builds on

43 of 75 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  11. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  12. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
31 more
  1. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  2. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  3. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  4. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  5. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  6. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  7. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 225
  8. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 197
  9. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  10. 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
  11. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103
  12. 2023Revisiting Relation Extraction in the era of Large Language Modelscited by 102
  13. 2019LassoNet: A Neural Network with Feature Sparsitycited by 82
  14. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 81
  15. 2024OpenGraph: Towards Open Graph Foundation Modelscited by 78
  16. 2024Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoningcited by 69
  17. 2018Effective Ways to Build and Evaluate Individual Survival Distributionscited by 64
  18. 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMscited by 61
  19. 2022LMPriors: Pre-Trained Language Models as Task-Specific Priorscited by 56
  20. 2023BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Modelscited by 53
  21. 2024DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literaturecited by 50
  22. 2017Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classificationcited by 39
  23. 2024LLM-Select: Feature Selection with Large Language Modelscited by 30
  24. 2024Facial Affective Behavior Analysis with Instruction Tuningcited by 26
  25. 2023Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidancecited by 24
  26. 2024Contextualization Distillation from Large Language Model for Knowledge Graph Completioncited by 18
  27. 2024Tabular Data Augmentation for Machine Learning: Progress and Prospects of Embracing Generative AIcited by 15
  28. 2024Tuning-Free Accountable Intervention for LLM Deployment -- A Metacognitive Approachcited by 11
  29. 2024ICE-SEARCH: A Language Model-Driven Feature Selection Approachcited by 8
  30. 2024Large Language Model Enhanced Knowledge Representation Learning: A Surveycited by 7
  31. 2020Correlated Differential Privacy: Feature Selection in Machine Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 13; 2026: 10.

27 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  2. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 125
  3. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 87
  4. 2024SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learningcited by 24
  5. 2025Large Language Model-based Data Science Agent: A Surveycited by 23
  6. 2025LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularizationcited by 10
  7. 2024Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Modelscited by 9
  8. 2025AUTOCT: Automating Interpretable Clinical Trial Prediction with LLM Agentscited by 9
  9. 2025LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessmentcited by 6
  10. 2025Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple LLM Judgescited by 3
  11. 2025LLM4FS: Leveraging Large Language Models for Feature Selectioncited by 2
  12. 2025Comprehend, Divide, and Conquer: Feature Subspace Exploration via Multi-Agent Hierarchical Reinforcement Learningcited by 1
15 more
  1. 2025Multi-level Diagnosis and Evaluation for Robust Tabular Feature Engineering with Large Language Modelscited by 1
  2. 2026LLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systemscited by 1
  3. 2025LAMP: Extracting Local Decision Surfaces From Large Language Modelscited by 0
  4. 2025PartnerMAS: An LLM Hierarchical Multi-Agent Framework for Business Partner Selection on High-Dimensional Featurescited by 0
  5. 2025LLM-FS-Agent: A Deliberative Role-based Large Language Model Architecture for Transparent Feature Selectioncited by 0
  6. 2025DeepFeature: LLM-Empowered Context-aware Feature Generation for Wearable Biosignalscited by 0
  7. 2026ReToP: Learning to Rewrite Electronic Health Records for Clinical Predictioncited by 0
  8. 2026Learning When to Trust LLM Priors: A Validated Framework for Semantic Prior Integrationcited by 0
  9. 2026Hippasus: Effective and Efficient Automatic Feature Augmentation for Machine Learning Tasks on Relational Datacited by 0
  10. 2026Detecting Jailbreak Attempts in Clinical Training LLMs Through Automated Linguistic Feature Extractioncited by 0
  11. 2026Same Input, Different Scores: A Multi Model Study on the Inconsistency of LLM Judgecited by 0
  12. 2026BoolXLLM: LLM-Assisted Explainability for Boolean Modelscited by 0
  13. 2026Feature Generation Using LLMs: An Evolutionary Algorithm Approachcited by 0
  14. 2026When Does a Classifier Help an LLM? Classifier-Guided Prompting and Hybrid Classifier-LLM Models for Credit-Default Predictioncited by 0
  15. 2026LLM-Enhanced Multi-Agent Reinforcement Learning for Unified Electric Vehicles-Charging Station-Grid Optimization in Public Charging Systemscited by 0