# Interactive Reinforcement Learning for Feature Selection with Decision Tree in the Loop

arXiv: https://arxiv.org/abs/2010.02506 · HTML: https://mlsanity.com/paper/2010.02506/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [Self-optimizing Feature Generation via Categorical Hashing Representation and Hierarchical Reinforcement Crossing](https://mlsanity.com/paper/2309.04612/) (2023)
- [Federated Generalized Category Discovery](https://mlsanity.com/paper/2305.14107/) (2023)
- [Deep Frequency Derivative Learning for Non-stationary Time Series Forecasting](https://mlsanity.com/paper/2407.00502/) (2024)
- [FedGCS: A Generative Framework for Efficient Client Selection in Federated Learning via Gradient-based Optimization](https://mlsanity.com/paper/2405.06312/) (2024)
- [UADB: Unsupervised Anomaly Detection Booster](https://mlsanity.com/paper/2306.01997/) (2023)
- [Machine Learning for Urban Air Quality Analytics: A Survey](https://mlsanity.com/paper/2310.09620/) (2023)
- [LLM Meeting Decision Trees on Tabular Data](https://mlsanity.com/paper/2505.17918/) (2025)
- [Feature Interaction Aware Automated Data Representation Transformation](https://mlsanity.com/paper/2309.17011/) (2023)
- [Autonomous Data Agents: A New Opportunity for Smart Data](https://mlsanity.com/paper/2509.18710/) (2025)
- [DELTA: Variational Disentangled Learning for Privacy-Preserving Data Reprogramming](https://mlsanity.com/paper/2509.00693/) (2025)
- [Feature and Instance Joint Selection: A Reinforcement Learning Perspective](https://mlsanity.com/paper/2205.07867/) (2022)
- [Toward Efficient Automated Feature Engineering](https://mlsanity.com/paper/2212.13152/) (2022)
- [Enhancing Classification Performance via Reinforcement Learning for Feature Selection](https://mlsanity.com/paper/2403.05979/) (2024)
- [Towards Data-Centric AI: A Comprehensive Survey of Traditional, Reinforcement, and Generative Approaches for Tabular Data Transformation](https://mlsanity.com/paper/2501.10555/) (2025)
- [Automation and Feature Selection Enhancement with Reinforcement Learning (RL)](https://mlsanity.com/paper/2503.11991/) (2025)
- [Dataforge: Agentic Platform for Autonomous Data Engineering](https://mlsanity.com/paper/2511.06185/) (2025)
- [CLAIRE: Compressed Latent Autoencoder for Industrial Representation and Evaluation -- A Deep Learning Framework for Smart Manufacturing](https://mlsanity.com/paper/2603.06361/) (2026)
- [Investigating Gender Fairness in Machine Learning-driven Personalized Care for Chronic Pain](https://mlsanity.com/paper/2402.19226/) (2024)

## Abstract (arXiv)

We study the problem of balancing effectiveness and efficiency in automated feature selection. After exploring many feature selection methods, we observe a computational dilemma: 1) traditional feature selection is mostly efficient, but difficult to identify the best subset; 2) the emerging reinforced feature selection automatically navigates to the best subset, but is usually inefficient. Can we bridge the gap between effectiveness and efficiency under automation? Motivated by this dilemma, we aim to develop a novel feature space navigation method. In our preliminary work, we leveraged interactive reinforcement learning to accelerate feature selection by external trainer-agent interaction. In this journal version, we propose a novel interactive and closed-loop architecture to simultaneously model interactive reinforcement learning (IRL) and decision tree feedback (DTF). Specifically, IRL is to create an interactive feature selection loop and DTF is to feed structured feature knowledge back to the loop. First, the tree-structured feature hierarchy from decision tree is leveraged to improve state representation. In particular, we represent the selected feature subset as an undirected graph of feature-feature correlations and a directed tree of decision features. We propose a new embedding method capable of empowering graph convolutional network to jointly learn state representation from both the graph and the tree. Second, the tree-structured feature hierarchy is exploited to develop a new reward scheme. In particular, we personalize reward assignment of agents based on decision tree feature importance. In addition, observing agents' actions can be feedback, we devise another reward scheme, to weigh and assign reward based on the feature selected frequency ratio in historical action records. Finally, we present extensive experiments on real-world datasets to show the improved performance.