# Feature Selection as Deep Sequential Generative Learning

arXiv: https://arxiv.org/abs/2403.03838 · HTML: https://mlsanity.com/paper/2403.03838/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 56 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DeepPINK: reproducible feature selection in deep neural networks](https://mlsanity.com/paper/1809.01185/) (2018)
- [Self-optimizing Feature Generation via Categorical Hashing Representation and Hierarchical Reinforcement Crossing](https://mlsanity.com/paper/2309.04612/) (2023)
- [Beyond Discrete Selection: Continuous Embedding Space Optimization for Generative Feature Selection](https://mlsanity.com/paper/2302.13221/) (2023)
- [Traceable Group-Wise Self-Optimizing Feature Transformation Learning: A Dual Optimization Perspective](https://mlsanity.com/paper/2306.16893/) (2023)
- [Efficient Reinforced Feature Selection via Early Stopping Traverse Strategy](https://mlsanity.com/paper/2109.14180/) (2021)
- [DIWIFT: Discovering Instance-wise Influential Features for Tabular Data](https://mlsanity.com/paper/2207.02773/) (2022)
- [Graph Convolutional Network-based Feature Selection for High-dimensional and Low-sample Size Data](https://mlsanity.com/paper/2211.14144/) (2022)

## Built on by (most cited first, 21 of 21)

- [Neuro-Symbolic Embedding for Short and Effective Feature Selection via Autoregressive Generation](https://mlsanity.com/paper/2404.17157/) (2024)
- [Topology-aware Reinforcement Feature Space Reconstruction for Graph Data](https://mlsanity.com/paper/2411.05742/) (2024)
- [GUME: Graphs and User Modalities Enhancement for Long-Tail Multimodal Recommendation](https://mlsanity.com/paper/2407.12338/) (2024)
- [Continuous Optimization for Feature Selection with Permutation-Invariant Embedding and Policy-Guided Search](https://mlsanity.com/paper/2505.11601/) (2025)
- [Teaching Time Series to See and Speak: Forecasting with Aligned Visual and Textual Perspectives](https://mlsanity.com/paper/2506.24124/) (2025)
- [Image-to-Image Translation with Diffusion Transformers and CLIP-Based Image Conditioning](https://mlsanity.com/paper/2505.16001/) (2025)
- [Autonomous Data Agents: A New Opportunity for Smart Data](https://mlsanity.com/paper/2509.18710/) (2025)
- [LLM-ML Teaming: Integrated Symbolic Decoding and Gradient Search for Valid and Stable Generative Feature Transformation](https://mlsanity.com/paper/2506.09085/) (2025)
- [Bridging the Domain Gap in Equation Distillation with Reinforcement Feedback](https://mlsanity.com/paper/2505.15572/) (2025)
- [Distribution Shift Aware Neural Tabular Learning](https://mlsanity.com/paper/2508.19486/) (2025)
- [DELTA: Variational Disentangled Learning for Privacy-Preserving Data Reprogramming](https://mlsanity.com/paper/2509.00693/) (2025)
- [Permutation-Invariant Representation Learning for Robust and Privacy-Preserving Feature Selection](https://mlsanity.com/paper/2510.05535/) (2025)
- [Enhancing Risk Assessment in Transformers with Loss-at-Risk Functions](https://mlsanity.com/paper/2411.02558/) (2024)
- [Towards Data-Centric AI: A Comprehensive Survey of Traditional, Reinforcement, and Generative Approaches for Tabular Data Transformation](https://mlsanity.com/paper/2501.10555/) (2025)
- [Biological Pathway Guided Gene Selection Through Collaborative Reinforcement Learning](https://mlsanity.com/paper/2505.24155/) (2025)
- [Hierarchical and Permutation-Invariant Feature Transformation Learning via Policy-Guided Embedding Search](https://mlsanity.com/paper/2609.10225/) (2026)
- [Building a Chinese Medical Dialogue System: Integrating Large-scale Corpora and Novel Models](https://mlsanity.com/paper/2410.03521/) (2024)
- [Data-Efficient Symbolic Regression via Foundation Model Distillation](https://mlsanity.com/paper/2508.19487/) (2025)
- [Dataforge: Agentic Platform for Autonomous Data Engineering](https://mlsanity.com/paper/2511.06185/) (2025)
- [Causally-Guided Diffusion for Stable Feature Selection](https://mlsanity.com/paper/2603.20930/) (2026)
- [Multi-Agent Procedural Graph Extraction with Structural and Logical Refinement](https://mlsanity.com/paper/2601.19170/) (2026)

## Abstract (arXiv)

Feature selection aims to identify the most pattern-discriminative feature subset. In prior literature, filter (e.g., backward elimination) and embedded (e.g., Lasso) methods have hyperparameters (e.g., top-K, score thresholding) and tie to specific models, thus, hard to generalize; wrapper methods search a feature subset in a huge discrete space and is computationally costly. To transform the way of feature selection, we regard a selected feature subset as a selection decision token sequence and reformulate feature selection as a deep sequential generative learning task that distills feature knowledge and generates decision sequences. Our method includes three steps: (1) We develop a deep variational transformer model over a joint of sequential reconstruction, variational, and performance evaluator losses. Our model can distill feature selection knowledge and learn a continuous embedding space to map feature selection decision sequences into embedding vectors associated with utility scores. (2) We leverage the trained feature subset utility evaluator as a gradient provider to guide the identification of the optimal feature subset embedding;(3) We decode the optimal feature subset embedding to autoregressively generate the best feature selection decision sequence with autostop. Extensive experimental results show this generative perspective is effective and generic, without large discrete search space and expert-specific hyperparameters.