# Inferring stochastic low-rank recurrent neural networks from neural data arXiv: https://arxiv.org/abs/2406.16749 ยท HTML: https://mlsanity.com/paper/2406.16749/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Builds on 29 of its 76 references that are in the map. - First posted on arXiv 24 Jun 2024. ## Builds on (most cited first, 29 of 29) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018) - [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019) - [Resurrecting Recurrent Neural Networks for Long Sequences](https://mlsanity.com/paper/2303.06349/) (2023) - [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020) - [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017) - [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017) - [Auto-Encoding Sequential Monte Carlo](https://mlsanity.com/paper/1705.10306/) (2017) - [Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity](https://mlsanity.com/paper/2109.04463/) (2021) - [Generalized Teacher Forcing for Learning Chaotic Dynamics](https://mlsanity.com/paper/2306.04406/) (2023) - [Linking connectivity, dynamics and computations in low-rank recurrent neural networks](https://mlsanity.com/paper/1711.09672/) (2017) - [Representation learning for neural population activity with Neural Data Transformers](https://mlsanity.com/paper/2108.01210/) (2021) - [Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE](https://mlsanity.com/paper/2011.04798/) (2020) - [Long Expressive Memory for Sequence Modeling](https://mlsanity.com/paper/2110.04744/) (2021) - [Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems](https://mlsanity.com/paper/2207.02542/) (2022) - [A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurements](https://mlsanity.com/paper/1612.07846/) (2016) - [SIXO: Smoothing Inference with Twisted Objectives](https://mlsanity.com/paper/2206.05952/) (2022) - [Bifurcations and loss jumps in RNN training](https://mlsanity.com/paper/2310.17561/) (2023) - [Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics](https://mlsanity.com/paper/2212.07892/) (2022) - [Streaming Variational Monte Carlo](https://mlsanity.com/paper/1906.01549/) (2019) - [Dynamics of random recurrent networks with correlated low-rank structure](https://mlsanity.com/paper/1909.04358/) (2019) - [Shaping dynamics with multiple populations in low-rank recurrent networks](https://mlsanity.com/paper/2007.02062/) (2020) - [Continuous Latent Process Flows](https://mlsanity.com/paper/2106.15580/) (2021) - [Expressive dynamics models with nonlinear injective readouts enable reliable recovery of latent features from neural activity](https://mlsanity.com/paper/2309.06402/) (2023) - [CORNN: Convex optimization of recurrent neural networks for rapid inference of neural dynamics](https://mlsanity.com/paper/2311.10200/) (2023) - [Trial matching: capturing variability with data-constrained spiking neural networks](https://mlsanity.com/paper/2306.03603/) (2023) - [Diversity of emergent dynamics in competitive threshold-linear networks](https://mlsanity.com/paper/1605.04463/) (2016) - [Fixed points of competitive threshold-linear networks](https://mlsanity.com/paper/1804.00794/) (2018) - [Resampling Gradients Vanish in Differentiable Sequential Monte Carlo Samplers](https://mlsanity.com/paper/2304.14390/) (2023) ## Built on by (most cited first, 15 of 15) - [True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics](https://mlsanity.com/paper/2505.13192/) (2025) - [Generative Modeling of Neural Dynamics via Latent Stochastic Differential Equations](https://mlsanity.com/paper/2412.12112/) (2024) - [Continuous-Time Piecewise-Linear Recurrent Neural Networks](https://mlsanity.com/paper/2602.15649/) (2026) - [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026) - [Neural timescales from a computational perspective](https://mlsanity.com/paper/2409.02684/) (2024) - [Detecting Invariant Manifolds in ReLU-Based RNNs](https://mlsanity.com/paper/2510.03814/) (2025) - [Modeling Neural Activity with Conditionally Linear Dynamical Systems](https://mlsanity.com/paper/2502.18347/) (2025) - [Double projection for reconstructing dynamical systems: between stochastic and deterministic regimes](https://mlsanity.com/paper/2510.01089/) (2025) - [A Factorized Low-Rank RNN Framework for Uncovering Independent Neural Latent Dynamics and Connectivity](https://mlsanity.com/paper/2511.13899/) (2025) - [The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modeling](https://mlsanity.com/paper/2605.31547/) (2026) - [Langevin Flows for Modeling Neural Latent Dynamics](https://mlsanity.com/paper/2507.11531/) (2025) - [JEDI: Jointly Embedded Inference of Neural Dynamics](https://mlsanity.com/paper/2603.10489/) (2026) - [Identifying Connectivity Distributions from Neural Dynamics Using Flows](https://mlsanity.com/paper/2603.26506/) (2026) - [Machine Learning Methods for Studying Latent Neural Activity Dynamics](https://mlsanity.com/paper/2606.10530/) (2026) - [A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems](https://mlsanity.com/paper/2607.14937/) (2026) ## Abstract (arXiv) A central aim in computational neuroscience is to relate the activity of large populations of neurons to an underlying dynamical system. Models of these neural dynamics should ideally be both interpretable and fit the observed data well. Low-rank recurrent neural networks (RNNs) exhibit such interpretability by having tractable dynamics. However, it is unclear how to best fit low-rank RNNs to data consisting of noisy observations of an underlying stochastic system. Here, we propose to fit stochastic low-rank RNNs with variational sequential Monte Carlo methods. We validate our method on several datasets consisting of both continuous and spiking neural data, where we obtain lower dimensional latent dynamics than current state of the art methods. Additionally, for low-rank models with piecewise linear nonlinearities, we show how to efficiently identify all fixed points in polynomial rather than exponential cost in the number of units, making analysis of the inferred dynamics tractable for large RNNs. Our method both elucidates the dynamical systems underlying experimental recordings and provides a generative model whose trajectories match observed variability.