# NAS-X: Neural Adaptive Smoothing via Twisting

arXiv: https://arxiv.org/abs/2308.14864 · HTML: https://mlsanity.com/paper/2308.14864/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 83 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Deep Kalman Filters](https://mlsanity.com/paper/1511.05121/) (2015)
- [Sequential Neural Models with Stochastic Layers](https://mlsanity.com/paper/1605.07571/) (2016)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Variational inference for Monte Carlo objectives](https://mlsanity.com/paper/1602.06725/) (2016)
- [Tighter Variational Bounds are Not Necessarily Better](https://mlsanity.com/paper/1802.04537/) (2018)
- [Reweighted Wake-Sleep](https://mlsanity.com/paper/1406.2751/) (2014)
- [Auto-Encoding Sequential Monte Carlo](https://mlsanity.com/paper/1705.10306/) (2017)
- [Elements of Sequential Monte Carlo](https://mlsanity.com/paper/1903.04797/) (2019)
- [Neural Adaptive Sequential Monte Carlo](https://mlsanity.com/paper/1506.03338/) (2015)
- [SIXO: Smoothing Inference with Twisted Objectives](https://mlsanity.com/paper/2206.05952/) (2022)
- [Nested Sequential Monte Carlo Methods](https://mlsanity.com/paper/1502.02536/) (2015)
- [Critic Sequential Monte Carlo](https://mlsanity.com/paper/2205.15460/) (2022)
- [Nested Variational Inference](https://mlsanity.com/paper/2106.11302/) (2021)
- [Energy-Inspired Models: Learning with Sampler-Induced Distributions](https://mlsanity.com/paper/1910.14265/) (2019)
- [Lookahead Strategies for Sequential Monte Carlo](https://mlsanity.com/paper/1302.5206/) (2013)
- [Learning Hard Alignments with Variational Inference](https://mlsanity.com/paper/1705.05524/) (2017)

## Built on by (most cited first, 4 of 4)

- [Bayesian Optimization of Antibodies Informed by a Generative Model of Evolving Sequences](https://mlsanity.com/paper/2412.07763/) (2024)
- [Robust Inverse Graphics via Probabilistic Inference](https://mlsanity.com/paper/2402.01915/) (2024)
- [Efficient Inference for Coupled Hidden Markov Models in Continuous Time and Discrete Space](https://mlsanity.com/paper/2510.12916/) (2025)
- [Neural Backward Filtering Forward Guiding](https://mlsanity.com/paper/2601.23030/) (2026)

## Abstract (arXiv)

Sequential latent variable models (SLVMs) are essential tools in statistics and machine learning, with applications ranging from healthcare to neuroscience. As their flexibility increases, analytic inference and model learning can become challenging, necessitating approximate methods. Here we introduce neural adaptive smoothing via twisting (NAS-X), a method that extends reweighted wake-sleep (RWS) to the sequential setting by using smoothing sequential Monte Carlo (SMC) to estimate intractable posterior expectations. Combining RWS and smoothing SMC allows NAS-X to provide low-bias and low-variance gradient estimates, and fit both discrete and continuous latent variable models. We illustrate the theoretical advantages of NAS-X over previous methods and explore these advantages empirically in a variety of tasks, including a challenging application to mechanistic models of neuronal dynamics. These experiments show that NAS-X substantially outperforms previous VI- and RWS-based methods in inference and model learning, achieving lower parameter error and tighter likelihood bounds.