# Recurrent Attentive Neural Process for Sequential Data

arXiv: https://arxiv.org/abs/1910.09323 · HTML: https://mlsanity.com/paper/1910.09323/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2019.

## Built on by (most cited first, 15 of 15)

- [Context-Aware Safe Reinforcement Learning for Non-Stationary Environments](https://mlsanity.com/paper/2101.00531/) (2021)
- [Convolutional conditional neural processes for local climate downscaling](https://mlsanity.com/paper/2101.07950/) (2021)
- [NP-Match: When Neural Processes meet Semi-Supervised Learning](https://mlsanity.com/paper/2207.01066/) (2022)
- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Graph Neural Processes for Spatio-Temporal Extrapolation](https://mlsanity.com/paper/2305.18719/) (2023)
- [CAMul: Calibrated and Accurate Multi-view Time-Series Forecasting](https://mlsanity.com/paper/2109.07438/) (2021)
- [When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecasting](https://mlsanity.com/paper/2106.03904/) (2021)
- [Reinforcement Learning in Presence of Discrete Markovian Context Evolution](https://mlsanity.com/paper/2202.06557/) (2022)
- [Functional optimal transport: map estimation and domain adaptation for functional data](https://mlsanity.com/paper/2102.03895/) (2021)
- [Social Processes: Self-Supervised Meta-Learning over Conversational Groups for Forecasting Nonverbal Social Cues](https://mlsanity.com/paper/2107.13576/) (2021)
- [Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies](https://mlsanity.com/paper/2310.10500/) (2023)
- [Meta-Learning for Koopman Spectral Analysis with Short Time-series](https://mlsanity.com/paper/2102.04683/) (2021)
- [NP-Match: Towards a New Probabilistic Model for Semi-Supervised Learning](https://mlsanity.com/paper/2301.13569/) (2023)
- [Multi-Task Neural Processes](https://mlsanity.com/paper/2110.14953/) (2021)
- [Machine Learning for Infectious Disease Risk Prediction: A Survey](https://mlsanity.com/paper/2308.03037/) (2023)

## Abstract (arXiv)

Neural processes (NPs) learn stochastic processes and predict the distribution of target output adaptively conditioned on a context set of observed input-output pairs. Furthermore, Attentive Neural Process (ANP) improved the prediction accuracy of NPs by incorporating attention mechanism among contexts and targets. In a number of real-world applications such as robotics, finance, speech, and biology, it is critical to learn the temporal order and recurrent structure from sequential data. However, the capability of NPs capturing these properties is limited due to its permutation invariance instinct. In this paper, we proposed the Recurrent Attentive Neural Process (RANP), or alternatively, Attentive Neural Process-RecurrentNeural Network(ANP-RNN), in which the ANP is incorporated into a recurrent neural network. The proposed model encapsulates both the inductive biases of recurrent neural networks and also the strength of NPs for modelling uncertainty. We demonstrate that RANP can effectively model sequential data and outperforms NPs and LSTMs remarkably in a 1D regression toy example as well as autonomous-driving applications.