# Concrete Autoencoders for Differentiable Feature Selection and Reconstruction

arXiv: https://arxiv.org/abs/1901.09346 · HTML: https://mlsanity.com/paper/1901.09346/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2019.

## Built on by (most cited first, 30 of 90)

- [Condensing Graphs via One-Step Gradient Matching](https://mlsanity.com/paper/2206.07746/) (2022)
- [Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning](https://mlsanity.com/paper/2201.03529/) (2022)
- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [Learning to Maximize Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2301.00557/) (2023)
- [Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling](https://mlsanity.com/paper/2107.02389/) (2021)
- [Locally Sparse Neural Networks for Tabular Biomedical Data](https://mlsanity.com/paper/2106.06468/) (2021)
- [A Performance-Driven Benchmark for Feature Selection in Tabular Deep Learning](https://mlsanity.com/paper/2311.05877/) (2023)
- [Estimating Conditional Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2306.03301/) (2023)
- [Interpretable Deep Clustering for Tabular Data](https://mlsanity.com/paper/2306.04785/) (2023)
- [MixupExplainer: Generalizing Explanations for Graph Neural Networks with Data Augmentation](https://mlsanity.com/paper/2307.07832/) (2023)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Adversarial Examples can be Effective Data Augmentation for Unsupervised Machine Learning](https://mlsanity.com/paper/2103.01895/) (2021)
- [Where to Pay Attention in Sparse Training for Feature Selection?](https://mlsanity.com/paper/2211.14627/) (2022)
- [Learning Topology-Specific Experts for Molecular Property Prediction](https://mlsanity.com/paper/2302.13693/) (2023)
- [End-to-end learnable EEG channel selection for deep neural networks with Gumbel-softmax](https://mlsanity.com/paper/2102.09050/) (2021)
- [Deep Unsupervised Feature Selection by Discarding Nuisance and Correlated Features](https://mlsanity.com/paper/2110.05306/) (2021)
- [Composite Feature Selection using Deep Ensembles](https://mlsanity.com/paper/2211.00631/) (2022)
- [Sequential Attention for Feature Selection](https://mlsanity.com/paper/2209.14881/) (2022)
- [RegExplainer: Generating Explanations for Graph Neural Networks in Regression Tasks](https://mlsanity.com/paper/2307.07840/) (2023)
- [LLMExplainer: Large Language Model based Bayesian Inference for Graph Explanation Generation](https://mlsanity.com/paper/2407.15351/) (2024)
- [Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments](https://mlsanity.com/paper/2506.00214/) (2025)
- [PSNet: Fast Data Structuring for Hierarchical Deep Learning on Point Cloud](https://mlsanity.com/paper/2205.14965/) (2022)
- [Interpretability with full complexity by constraining feature information](https://mlsanity.com/paper/2211.17264/) (2022)
- [Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks](https://mlsanity.com/paper/2303.07200/) (2023)
- [Deep Feature Screening: Feature Selection for Ultra High-Dimensional Data via Deep Neural Networks](https://mlsanity.com/paper/2204.01682/) (2022)
- [GCondNet: A Novel Method for Improving Neural Networks on Small High-Dimensional Tabular Data](https://mlsanity.com/paper/2211.06302/) (2022)
- [Weight Predictor Network with Feature Selection for Small Sample Tabular Biomedical Data](https://mlsanity.com/paper/2211.15616/) (2022)
- [UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection](https://mlsanity.com/paper/2207.03809/) (2022)
- [Multi-modal Differentiable Unsupervised Feature Selection](https://mlsanity.com/paper/2303.09381/) (2023)
- [Differentiable Feature Selection, a Reparameterization Approach](https://mlsanity.com/paper/2107.10030/) (2021)

## Abstract (arXiv)

We introduce the concrete autoencoder, an end-to-end differentiable method for global feature selection, which efficiently identifies a subset of the most informative features and simultaneously learns a neural network to reconstruct the input data from the selected features. Our method is unsupervised, and is based on using a concrete selector layer as the encoder and using a standard neural network as the decoder. During the training phase, the temperature of the concrete selector layer is gradually decreased, which encourages a user-specified number of discrete features to be learned. During test time, the selected features can be used with the decoder network to reconstruct the remaining input features. We evaluate concrete autoencoders on a variety of datasets, where they significantly outperform state-of-the-art methods for feature selection and data reconstruction. In particular, on a large-scale gene expression dataset, the concrete autoencoder selects a small subset of genes whose expression levels can be use to impute the expression levels of the remaining genes. In doing so, it improves on the current widely-used expert-curated L1000 landmark genes, potentially reducing measurement costs by 20%. The concrete autoencoder can be implemented by adding just a few lines of code to a standard autoencoder.