# Combining Generative and Discriminative Models for Hybrid Inference

arXiv: https://arxiv.org/abs/1906.02547 · HTML: https://mlsanity.com/paper/1906.02547/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2019.

## Built on by (most cited first, 30 of 34)

- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Towards Multi-spatiotemporal-scale Generalized PDE Modeling](https://mlsanity.com/paper/2209.15616/) (2022)
- [KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics](https://mlsanity.com/paper/2107.10043/) (2021)
- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learning](https://mlsanity.com/paper/2203.16797/) (2022)
- [Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization](https://mlsanity.com/paper/2205.02640/) (2022)
- [The Expressive Power of Graph Neural Networks: A Survey](https://mlsanity.com/paper/2308.08235/) (2023)
- [Hybrid Predictive Coding: Inferring, Fast and Slow](https://mlsanity.com/paper/2204.02169/) (2022)
- [DANSE: Data-driven Non-linear State Estimation of Model-free Process in Unsupervised Learning Setup](https://mlsanity.com/paper/2306.03897/) (2023)
- [Unsupervised Learned Kalman Filtering](https://mlsanity.com/paper/2110.09005/) (2021)
- [Neural Structured Prediction for Inductive Node Classification](https://mlsanity.com/paper/2204.07524/) (2022)
- [Knowledge-augmented Deep Learning and Its Applications: A Survey](https://mlsanity.com/paper/2212.00017/) (2022)
- [Neural Augmentation of Kalman Filter with Hypernetwork for Channel Tracking](https://mlsanity.com/paper/2109.12561/) (2021)
- [Learning Modular Simulations for Homogeneous Systems](https://mlsanity.com/paper/2210.16294/) (2022)
- [AmGCL: Feature Imputation of Attribute Missing Graph via Self-supervised Contrastive Learning](https://mlsanity.com/paper/2305.03741/) (2023)
- [Self-Supervised Inference in State-Space Models](https://mlsanity.com/paper/2107.13349/) (2021)
- [Diffusion Probabilistic Models for Structured Node Classification](https://mlsanity.com/paper/2302.10506/) (2023)
- [Likelihood Annealing: Fast Calibrated Uncertainty for Regression](https://mlsanity.com/paper/2302.11012/) (2023)
- [AI-Aided Kalman Filters](https://mlsanity.com/paper/2410.12289/) (2024)
- [Learning Dynamics and Structure of Complex Systems Using Graph Neural Networks](https://mlsanity.com/paper/2202.10996/) (2022)
- [Physics-Informed Data Denoising for Real-Life Sensing Systems](https://mlsanity.com/paper/2311.06968/) (2023)
- [Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptation](https://mlsanity.com/paper/2501.00873/) (2025)
- [Deep Variational Sequential Monte Carlo for High-Dimensional Observations](https://mlsanity.com/paper/2501.05982/) (2025)
- [Green MLOps to Green GenOps: An Empirical Study of Energy Consumption in Discriminative and Generative AI Operations](https://mlsanity.com/paper/2503.23934/) (2025)
- [Continual Learning of Multi-modal Dynamics with External Memory](https://mlsanity.com/paper/2203.00936/) (2022)
- [Semi-Supervised Model-Free Bayesian State Estimation from Compressed Measurements](https://mlsanity.com/paper/2407.07368/) (2024)
- [Phy-Taylor: Physics-Model-Based Deep Neural Networks](https://mlsanity.com/paper/2209.13511/) (2022)
- [Classification-Aided Robust Multiple Target Tracking Using Neural Enhanced Message Passing](https://mlsanity.com/paper/2310.12407/) (2023)
- [OptiState: State Estimation of Legged Robots using Gated Networks with Transformer-based Vision and Kalman Filtering](https://mlsanity.com/paper/2401.16719/) (2024)
- [pDANSE: Particle-based Data-driven Nonlinear State Estimation from Nonlinear Measurements](https://mlsanity.com/paper/2510.27503/) (2025)

## Abstract (arXiv)

A graphical model is a structured representation of the data generating process. The traditional method to reason over random variables is to perform inference in this graphical model. However, in many cases the generating process is only a poor approximation of the much more complex true data generating process, leading to suboptimal estimation. The subtleties of the generative process are however captured in the data itself and we can `learn to infer', that is, learn a direct mapping from observations to explanatory latent variables. In this work we propose a hybrid model that combines graphical inference with a learned inverse model, which we structure as in a graph neural network, while the iterative algorithm as a whole is formulated as a recurrent neural network. By using cross-validation we can automatically balance the amount of work performed by graphical inference versus learned inference. We apply our ideas to the Kalman filter, a Gaussian hidden Markov model for time sequences, and show, among other things, that our model can estimate the trajectory of a noisy chaotic Lorenz Attractor much more accurately than either the learned or graphical inference run in isolation.