# Fully differentiable model discovery

arXiv: https://arxiv.org/abs/2106.04886 · HTML: https://mlsanity.com/paper/2106.04886/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Digital Twin: Values, Challenges and Enablers](https://mlsanity.com/paper/1910.01719/) (2019)
- [DeepMoD: Deep learning for Model Discovery in noisy data](https://mlsanity.com/paper/1904.09406/) (2019)
- [Data-driven discovery of PDEs in complex datasets](https://mlsanity.com/paper/1808.10788/) (2018)
- [A Unified Framework for Sparse Relaxed Regularized Regression: SR3](https://mlsanity.com/paper/1807.05411/) (2018)
- [On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression](https://mlsanity.com/paper/2012.01937/) (2020)
- [Temporal Normalizing Flows](https://mlsanity.com/paper/1912.09092/) (2019)
- [Learning physically consistent mathematical models from data using group sparsity](https://mlsanity.com/paper/2012.06391/) (2020)
- [Machine Discovery of Partial Differential Equations from Spatiotemporal Data](https://mlsanity.com/paper/1909.06730/) (2019)
- [Sparsely constrained neural networks for model discovery of PDEs](https://mlsanity.com/paper/2011.04336/) (2020)

## Built on by (most cited first, 1 of 1)

- [Variational inference of fractional Brownian motion with linear computational complexity](https://mlsanity.com/paper/2203.07961/) (2022)

## Abstract (arXiv)

Model discovery aims at autonomously discovering differential equations underlying a dataset. Approaches based on Physics Informed Neural Networks (PINNs) have shown great promise, but a fully-differentiable model which explicitly learns the equation has remained elusive. In this paper we propose such an approach by integrating neural network-based surrogates with Sparse Bayesian Learning (SBL). This combination yields a robust model discovery algorithm, which we showcase on various datasets. We then identify a connection with multitask learning, and build on it to construct a Physics Informed Normalizing Flow (PINF). We present a proof-of-concept using a PINF to directly learn a density model from single particle data. Our work expands PINNs to various types of neural network architectures, and connects neural network-based surrogates to the rich field of Bayesian parameter inference.