# Neuronal Stochastic Attention Circuit (NSAC) for Probabilistic Representation Learning

arXiv: https://arxiv.org/abs/2605.26061 · HTML: https://mlsanity.com/paper/2605.26061/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 25 May 2026.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates](https://mlsanity.com/paper/2008.10546/) (2020)
- [Closed-form Continuous-time Neural Models](https://mlsanity.com/paper/2106.13898/) (2021)
- [ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling](https://mlsanity.com/paper/2402.10635/) (2024)
- [HUST bearing: a practical dataset for ball bearing fault diagnosis](https://mlsanity.com/paper/2302.12533/) (2023)
- [Continuous-Time Attention: PDE-Guided Mechanisms for Long-Sequence Transformers](https://mlsanity.com/paper/2505.20666/) (2025)
- [CARLE: A Hybrid Deep-Shallow Learning Framework for Robust and Explainable RUL Estimation of Rolling Element Bearings](https://mlsanity.com/paper/2510.17846/) (2025)
- [Neuronal Circuit Policies](https://mlsanity.com/paper/1803.08554/) (2018)
- [Neuronal Attention Circuit (NAC) for Representation Learning](https://mlsanity.com/paper/2512.10282/) (2025)

## Built on by (most cited first, 1 of 1)

- [Continuous-Time Machine Learning: A Unified Mathematical Perspective](https://mlsanity.com/paper/2609.16710/) (2026)

## Abstract (arXiv)

Reliable uncertainty quantification in continuous-time (CT) representation learning remains nascent, particularly within CT attention literature. We introduce the Neuronal Stochastic Attention Circuit (NSAC), a novel biologically-inspired CT attention architecture that reformulates attention logit computation as the solution of an Ornstein-Uhlenbeck stochastic differential equation modulated by input-dependent, nonlinear interlinked gates derived from repurposed C. elegans Neuronal Circuit Policies (NCPs) wiring mechanism. It induces a Gaussian distribution over logits that propagates principled stochasticity through a logistic-normal distribution over attention weights to yield probabilistic output. A two-term objective function combining Gaussian negative log-likelihood with an epistemic-separation regularizer enforces higher predictive variance under distributional shifts and enables joint quantification of aleatoric and epistemic uncertainty. Theoretically, we provide: (i) state stability bounds; (ii) closed-form guarantees; and (iii) frozen-coefficient error approximation. Empirically, we implement NSAC in a diverse set of learning tasks including: (i) irregular CT function approximation; (ii) multivariate regression; (iii) long-range forecasting; (iv) Industry 4.0; and (v) lane-keeping of autonomous vehicles. We observe that NSAC remains competitive against several baselines in terms of accuracy and produces informative uncertainty estimates while being interpretable at the neuronal cell level.