# Hamiltonian Monte Carlo Particle Swarm Optimizer

arXiv: https://arxiv.org/abs/2206.14134 · HTML: https://mlsanity.com/paper/2206.14134/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 8 May 2022.

## Builds on (most cited first, 5 of 5)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Disentangling Adaptive Gradient Methods from Learning Rates](https://mlsanity.com/paper/2002.11803/) (2020)
- [AdaSwarm: Augmenting Gradient-Based optimizers in Deep Learning with Swarm Intelligence](https://mlsanity.com/paper/2006.09875/) (2020)
- [Bayesian interpretation of SGD as Ito process](https://mlsanity.com/paper/1911.09011/) (2019)
- [A Swarm Variant for the Schr"odinger Solver](https://mlsanity.com/paper/2104.04795/) (2021)

## Abstract (arXiv)

We introduce the Hamiltonian Monte Carlo Particle Swarm Optimizer (HMC-PSO), an optimization algorithm that reaps the benefits of both Exponentially Averaged Momentum PSO and HMC sampling. The coupling of the position and velocity of each particle with Hamiltonian dynamics in the simulation allows for extensive freedom for exploration and exploitation of the search space. It also provides an excellent technique to explore highly non-convex functions while ensuring efficient sampling. We extend the method to approximate error gradients in closed form for Deep Neural Network (DNN) settings. We discuss possible methods of coupling and compare its performance to that of state-of-the-art optimizers on the Golomb's Ruler problem and Classification tasks.