# On the challenges of learning with inference networks on sparse, high-dimensional data

arXiv: https://arxiv.org/abs/1710.06085 · HTML: https://mlsanity.com/paper/1710.06085/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2017.

## Built on by (most cited first, 21 of 21)

- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Amortized Variational Inference: When and Why?](https://mlsanity.com/paper/2307.11018/) (2023)
- [Preconditioned training of normalizing flows for variational inference in inverse problems](https://mlsanity.com/paper/2101.03709/) (2021)
- [Amortized Variational Inference: A Systematic Review](https://mlsanity.com/paper/2209.10888/) (2022)
- [Stochastic Iterative Graph Matching](https://mlsanity.com/paper/2106.02206/) (2021)
- [Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes](https://mlsanity.com/paper/2302.04534/) (2023)
- [Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling](https://mlsanity.com/paper/2104.05064/) (2021)
- [Variational Laplace Autoencoders](https://mlsanity.com/paper/2211.17267/) (2022)
- [Reducing the Amortization Gap in Variational Autoencoders: A Bayesian Random Function Approach](https://mlsanity.com/paper/2102.03151/) (2021)
- [TabRep: Training Tabular Diffusion Models with a Simple and Effective Continuous Representation](https://mlsanity.com/paper/2504.04798/) (2025)
- [CryoSPIN: Improving Ab-Initio Cryo-EM Reconstruction with Semi-Amortized Pose Inference](https://mlsanity.com/paper/2406.10455/) (2024)
- [Unbiased Learning of Deep Generative Models with Structured Discrete Representations](https://mlsanity.com/paper/2306.08230/) (2023)
- [ParamReL: Learning Parameter Space Representation via Progressively Encoding Bayesian Flow Networks](https://mlsanity.com/paper/2405.15268/) (2024)
- [A Stochastic Time Series Model for Predicting Financial Trends using NLP](https://mlsanity.com/paper/2102.01290/) (2021)
- [Hyperspectral Image Compression Using Implicit Neural Representation](https://mlsanity.com/paper/2302.04129/) (2023)
- [Compressive Features in Offline Reinforcement Learning for Recommender Systems](https://mlsanity.com/paper/2111.08817/) (2021)
- [Prior Density Learning in Variational Bayesian Phylogenetic Parameters Inference](https://mlsanity.com/paper/2302.02522/) (2023)
- [When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination](https://mlsanity.com/paper/2402.15283/) (2024)
- [Mapping of Lesion Images to Somatic Mutations](https://mlsanity.com/paper/2512.02162/) (2025)
- [Iterative Amortized Hierarchical VAE](https://mlsanity.com/paper/2601.15894/) (2026)
- [Comment-level Topic Drift Analysis in the Reddit Corpus](https://mlsanity.com/paper/2608.19133/) (2026)

## Abstract (arXiv)

We study parameter estimation in Nonlinear Factor Analysis (NFA) where the generative model is parameterized by a deep neural network. Recent work has focused on learning such models using inference (or recognition) networks; we identify a crucial problem when modeling large, sparse, high-dimensional datasets -- underfitting. We study the extent of underfitting, highlighting that its severity increases with the sparsity of the data. We propose methods to tackle it via iterative optimization inspired by stochastic variational inference citephoffman2013stochastic and improvements in the sparse data representation used for inference. The proposed techniques drastically improve the ability of these powerful models to fit sparse data, achieving state-of-the-art results on a benchmark text-count dataset and excellent results on the task of top-N recommendation.