Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learning
Not on the map. Why?
Abstract (arXiv)
Modelling the behaviour of highly nonlinear dynamical systems with robust uncertainty quantification is a challenging task which typically requires approaches specifically designed to address the problem at hand. We introduce a domain-agnostic model to address this issue termed the deep latent force model (DLFM), a deep Gaussian process with physics-informed kernels at each layer, derived from ordinary differential equations using the framework of process convolutions. Two distinct formulations of the DLFM are presented which utilise weight-space and variational inducing points-based Gaussian process approximations, both of which are amenable to doubly stochastic variational inference. We present empirical evidence of the capability of the DLFM to capture the dynamics present in highly nonlinear real-world multi-output time series data. Additionally, we find that the DLFM is capable of achieving comparable performance to a range of non-physics-informed probabilistic models on benchmark univariate regression tasks. We also empirically assess the negative impact of the inducing points framework on the extrapolation capabilities of LFM-based models.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 61 references that are in the map. First posted on arXiv 24 Nov 2023.
Builds on
34 of 61 references.
References · 34
- 2017Attention Is All You Needcited by 60,361
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
- 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
- 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 609
- 2015Variational Dropout and the Local Reparameterization Trickcited by 490
- 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
- 2013Gaussian Processes for Big Datacited by 383
- 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
- 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 238
22 more
- 2017Doubly Stochastic Variational Inference for Deep Gaussian Processescited by 143
- 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 127
- 2014Avoiding pathologies in very deep networkscited by 33
- 2016Random Feature Expansions for Deep Gaussian Processescited by 32
- 2018Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlocited by 32
- 2018Deep convolutional Gaussian processescited by 21
- 2019Deep Gaussian Processes with Importance-Weighted Variational Inferencecited by 19
- 2019Implicit Posterior Variational Inference for Deep Gaussian Processescited by 17
- 2022Markovian Gaussian Process Variational Autoencoderscited by 16
- 2020Physics Informed Deep Kernel Learningcited by 12
- 2018Non-linear process convolutions for multi-output Gaussian processescited by 10
- 2019Efficient Amortised Bayesian Inference for Hierarchical and Nonlinear Dynamical Systemscited by 10
- 2019Black-Box Inference for Non-Linear Latent Force Modelscited by 9
- 2020Inter-domain Deep Gaussian Processescited by 9
- 2019Compositional uncertainty in deep Gaussian processescited by 8
- 2021Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Featurescited by 8
- 2022On the inability of Gaussian process regression to optimally learn compositional functionscited by 7
- 2018Constraining the Dynamics of Deep Probabilistic Modelscited by 5
- 2018Fast Kernel Approximations for Latent Force Models and Convolved Multiple-Output Gaussian processescited by 5
- 2021Learning Nonparametric Volterra Kernels with Gaussian Processescited by 4
- 2021Approximate Latent Force Model Inferencecited by 1
- 2022Shallow and Deep Nonparametric Convolutions for Gaussian Processescited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.