# Generator Assisted Mixture of Experts For Feature Acquisition in Batch

arXiv: https://arxiv.org/abs/2312.12574 · HTML: https://mlsanity.com/paper/2312.12574/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 21 of 21)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training](https://mlsanity.com/paper/2103.00123/) (2021)
- [Lazier Than Lazy Greedy](https://mlsanity.com/paper/1409.7938/) (2014)
- [EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE](https://mlsanity.com/paper/1809.11142/) (2018)
- [Learning From Less Data: A Unified Data Subset Selection and Active Learning Framework for Computer Vision](https://mlsanity.com/paper/1901.01151/) (2019)
- [Active Feature Acquisition with Generative Surrogate Models](https://mlsanity.com/paper/2010.02433/) (2020)
- [Restricted Strong Convexity Implies Weak Submodularity](https://mlsanity.com/paper/1612.00804/) (2016)
- [Classification Under Human Assistance](https://mlsanity.com/paper/2006.11845/) (2020)
- [Classification with Costly Features using Deep Reinforcement Learning](https://mlsanity.com/paper/1711.07364/) (2017)
- [Active Learning under Label Shift](https://mlsanity.com/paper/2007.08479/) (2020)
- [Submodular Maximization Beyond Non-negativity: Guarantees, Fast Algorithms, and Applications](https://mlsanity.com/paper/1904.09354/) (2019)
- [Scalable Greedy Feature Selection via Weak Submodularity](https://mlsanity.com/paper/1703.02723/) (2017)
- [Active Feature Acquisition with Supervised Matrix Completion](https://mlsanity.com/paper/1802.05380/) (2018)
- [Adaptive Feature Selection: Computationally Efficient Online Sparse Linear Regression under RIP](https://mlsanity.com/paper/1706.04690/) (2017)
- [Online Sparse Linear Regression](https://mlsanity.com/paper/1603.02250/) (2016)
- [Dynamic Feature Acquisition with Arbitrary Conditional Flows](https://mlsanity.com/paper/2006.07701/) (2020)
- [Scalable and Accurate Online Feature Selection for Big Data](https://mlsanity.com/paper/1511.09263/) (2015)
- [Sample Efficient Stochastic Gradient Iterative Hard Thresholding Method for Stochastic Sparse Linear Regression with Limited Attribute Observation](https://mlsanity.com/paper/1809.01765/) (2018)

## Built on by (most cited first, 2 of 2)

- [A Survey on Active Feature Acquisition Strategies](https://mlsanity.com/paper/2502.11067/) (2025)
- [Cohort-Based Active Modality Acquisition](https://mlsanity.com/paper/2505.16791/) (2025)

## Abstract (arXiv)

Given a set of observations, feature acquisition is about finding the subset of unobserved features which would enhance accuracy. Such problems have been explored in a sequential setting in prior work. Here, the model receives feedback from every new feature acquired and chooses to explore more features or to predict. However, sequential acquisition is not feasible in some settings where time is of the essence. We consider the problem of feature acquisition in batch, where the subset of features to be queried in batch is chosen based on the currently observed features, and then acquired as a batch, followed by prediction. We solve this problem using several technical innovations. First, we use a feature generator to draw a subset of the synthetic features for some examples, which reduces the cost of oracle queries. Second, to make the feature acquisition problem tractable for the large heterogeneous observed features, we partition the data into buckets, by borrowing tools from locality sensitive hashing and then train a mixture of experts model. Third, we design a tractable lower bound of the original objective. We use a greedy algorithm combined with model training to solve the underlying problem. Experiments with four datasets show that our approach outperforms these methods in terms of trade-off between accuracy and feature acquisition cost.