# A Hardware-oriented Approach for Efficient Bayesian Inference Computation and Deployment

arXiv: https://arxiv.org/abs/2607.17855 · HTML: https://mlsanity.com/paper/2607.17855/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 20 Jul 2026.

## Builds on (most cited first, 20 of 20)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1802.05814/) (2018)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [pymdp: A Python library for active inference in discrete state spaces](https://mlsanity.com/paper/2201.03904/) (2022)
- [Patterns of Scalable Bayesian Inference](https://mlsanity.com/paper/1602.05221/) (2016)
- [End-to-End Pixel-Based Deep Active Inference for Body Perception and Action](https://mlsanity.com/paper/2001.05847/) (2019)
- [Format Abstraction for Sparse Tensor Algebra Compilers](https://mlsanity.com/paper/1804.10112/) (2018)
- [A Novel Adaptive Controller for Robot Manipulators based on Active Inference](https://mlsanity.com/paper/1909.12768/) (2019)
- [PGMax: Factor Graphs for Discrete Probabilistic Graphical Models and Loopy Belief Propagation in JAX](https://mlsanity.com/paper/2202.04110/) (2022)
- [Resolving uncertainty on the fly: Modeling adaptive driving behavior as active inference](https://mlsanity.com/paper/2311.06417/) (2023)
- [Evaluating Emerging AI/ML Accelerators: IPU, RDU, and NVIDIA/AMD GPUs](https://mlsanity.com/paper/2311.04417/) (2023)
- [Towards smart and adaptive agents for active sensing on edge devices](https://mlsanity.com/paper/2501.06262/) (2025)
- [Probabilistic Inference in the Era of Tensor Networks and Differential Programming](https://mlsanity.com/paper/2405.14060/) (2024)
- [Mobile Manipulation with Active Inference for Long-Horizon Rearrangement Tasks](https://mlsanity.com/paper/2507.17338/) (2025)
- [Evaluation of "As-Intended" Vehicle Dynamics using the Active Inference Framework](https://mlsanity.com/paper/2506.00035/) (2025)
- [Embedded FPGA Acceleration of Brain-Like Neural Networks: Online Learning to Scalable Inference](https://mlsanity.com/paper/2506.18530/) (2025)

## Abstract (arXiv)

Bayesian inference provides a principled foundation for reasoning under uncertainty, but its computational cost hinders deployment on resource-constrained edge devices. In this paper, we present a hardware-oriented methodology for accelerating discrete Bayesian inference on commercial off-the-shelf embedded GPUs. We identify that the latency of a broad class of variational message-passing algorithms is dominated by tensor contractions. Our approach restructures the memory layout of these operations using two complementary merging strategies that produce compact, regularly-shaped primitives better suited for efficient GPU execution. We then introduce optional sparse array representations and a tensor-clustering scheme to reduce the memory footprint. We instantiate the methodology and produce optimized variants of three message-passing algorithms for Hidden Markov Models (HMMs), namely variational filtering, variational message passing, and marginal message passing. Furthermore, we complement this with a machine-learning-based autotuner that automatically selects the best-performing algorithmic variant for a given generative model specification. Benchmarked on an NVIDIA Jetson Orin AGX across 770 randomly sampled realistic Partially Observable Markov Decision Process (POMDP) configurations, our implementations achieve speedups of up to 5x, with typical gains of 2-2.5x, while producing numerically identical outputs to the baseline implementations.