# Discrete Latent Structure in Neural Networks

arXiv: https://arxiv.org/abs/2301.07473 · HTML: https://mlsanity.com/paper/2301.07473/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2023.

## Built on by (most cited first, 5 of 5)

- [Graph Deep Learning for Time Series Forecasting](https://mlsanity.com/paper/2310.15978/) (2023)
- [Relational Conformal Prediction for Correlated Time Series](https://mlsanity.com/paper/2502.09443/) (2025)
- [Conditional computation in neural networks: principles and research trends](https://mlsanity.com/paper/2403.07965/) (2024)
- [Graph-based Virtual Sensing from Sparse and Partial Multivariate Observations](https://mlsanity.com/paper/2402.12598/) (2024)
- [Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference](https://mlsanity.com/paper/2312.10193/) (2023)

## Abstract (arXiv)

Many types of data from fields including natural language processing, computer vision, and bioinformatics, are well represented by discrete, compositional structures such as trees, sequences, or matchings. Latent structure models are a powerful tool for learning to extract such representations, offering a way to incorporate structural bias, discover insight about the data, and interpret decisions. However, effective training is challenging, as neural networks are typically designed for continuous computation. This text explores three broad strategies for learning with discrete latent structure: continuous relaxation, surrogate gradients, and probabilistic estimation. Our presentation relies on consistent notations for a wide range of models. As such, we reveal many new connections between latent structure learning strategies, showing how most consist of the same small set of fundamental building blocks, but use them differently, leading to substantially different applicability and properties.