mlsanityData to 7 Oct 2026

Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Models

arXiv 2410.01795 ↗First posted 2 Oct 2024cs.LG cs.CL q-bio.GNcited by 9

Not on the map. Why?

Abstract (arXiv)

Predicting phenotypes with complex genetic bases based on a small, interpretable set of variant features remains a challenging task. Conventionally, data-driven approaches are utilized for this task, yet the high dimensional nature of genotype data makes the analysis and prediction difficult. Motivated by the extensive knowledge encoded in pre-trained LLMs and their success in processing complex biomedical concepts, we set to examine the ability of LLMs in feature selection and engineering for tabular genotype data, with a novel knowledge-driven framework. We develop FREEFORM, Free-flow Reasoning and Ensembling for Enhanced Feature Output and Robust Modeling, designed with chain-of-thought and ensembling principles, to select and engineer features with the intrinsic knowledge of LLMs. Evaluated on two distinct genotype-phenotype datasets, genetic ancestry and hereditary hearing loss, we find this framework outperforms several data-driven methods, particularly on low-shot regimes. FREEFORM is available as open-source framework at GitHub: https://github.com/PennShenLab/FREEFORM.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 27 of its 49 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

27 of 49 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  10. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  11. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  12. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
15 more
  1. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  2. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  3. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
  4. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  5. 2023Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineeringcited by 125
  6. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Informationcited by 115
  7. 2024Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learningcited by 79
  8. 2024Large language models surpass human experts in predicting neuroscience resultscited by 69
  9. 2024Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoningcited by 69
  10. 2022LMPriors: Pre-Trained Language Models as Task-Specific Priorscited by 56
  11. 2024DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literaturecited by 50
  12. 2024LLM-Select: Feature Selection with Large Language Modelscited by 29
  13. 2024Exploring Large Language Models for Feature Selection: A Data-centric Perspectivecited by 27
  14. 2024ICE-SEARCH: A Language Model-Driven Feature Selection Approachcited by 8
  15. 2024GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databasescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 2025From Promising Capability to Pervasive Bias: Assessing Large Language Models for Emergency Department Triagecited by 5
  2. 2025LLM4FS: Leveraging Large Language Models for Feature Selectioncited by 2
  3. 2025Enabling Few-Shot Alzheimer's Disease Diagnosis on Biomarker Data with Tabular LLMscited by 1
  4. 2026LLM-FS: Zero-Shot Feature Selection for Effective and Interpretable Malware Detectioncited by 1
  5. 2025A Systematic Review on the Generative AI Applications in Human Medical Genomicscited by 0
  6. 2025PartnerMAS: An LLM Hierarchical Multi-Agent Framework for Business Partner Selection on High-Dimensional Featurescited by 0
  7. 2026Tabular LLMs for Interpretable Few-Shot Alzheimer's Disease Prediction with Multimodal Biomedical Datacited by 0
  8. 2026Mamba-SSM with LLM Reasoning for Feature Selection: Faithfulness-Aware Biomarker Discoverycited by 0
  9. 2026LATTEArena: An Evaluation Framework for LLM-powered Tabular Feature Engineering (Extended Version)cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers
  2. 2025Tabular Feature Discovery With Reasoning Type Exploration
  3. 2024Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning
  4. 2025How Effectively Can Large Language Models Connect SNP Variants and ECG Phenotypes for Cardiovascular Risk Prediction?
  5. 2026Feature Generation Using LLMs: An Evolutionary Algorithm Approach
  6. 2024Geneverse: A collection of Open-source Multimodal Large Language Models for Genomic and Proteomic Research
  7. 2026SNPgen: Phenotype-Supervised Genotype Representation and Synthetic Data Generation via Latent Diffusion
  8. 2025Gene-R1: Reasoning with Data-Augmented Lightweight LLMs for Gene Set Analysis