# Refined Gradient-Based Temperature Optimization for the Replica-Exchange Monte-Carlo Method

arXiv: https://arxiv.org/abs/2601.13542 · HTML: https://mlsanity.com/paper/2601.13542/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 33 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Builds on (most cited first, 6 of 6)

- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Automatic Differentiation of Programs with Discrete Randomness](https://mlsanity.com/paper/2210.08572/) (2022)
- [Differentiating Metropolis-Hastings to Optimize Intractable Densities](https://mlsanity.com/paper/2306.07961/) (2023)
- [Effects of setting the temperatures in the parallel tempering Monte Carlo algorithm](https://mlsanity.com/paper/1907.03906/) (2019)
- [Convergence Acceleration of Markov Chain Monte Carlo-based Gradient Descent by Deep Unfolding](https://mlsanity.com/paper/2402.13608/) (2024)
- [Policy Gradients for Optimal Parallel Tempering MCMC](https://mlsanity.com/paper/2409.01574/) (2024)

## Abstract (arXiv)

The replica-exchange Monte-Carlo (RXMC) method is a powerful Markov-chain Monte-Carlo algorithm for sampling from multi-modal distributions, which are challenging for conventional methods. The sampling efficiency of the RXMC method depends highly on the selection of the temperatures, and finding optimal temperatures remains a challenge. In this study, we propose a refined online temperature selection method by extending the gradient-based optimization framework proposed previously. Building upon the existing temperature update approach, we introduce a reparameterization technique to strictly enforce physical constraints, such as the monotonic ordering of inverse temperatures, which were not explicitly addressed in the original formulation. The proposed method defines the variance of acceptance rates between adjacent replicas as a loss function, estimates its gradient using differential information from the sampling process, and optimizes the temperatures via gradient descent. We demonstrate the effectiveness of our method through experiments on benchmark spin systems, including the two-dimensional ferromagnetic Ising model, the two-dimensional ferromagnetic XY model, and the three-dimensional Edwards-Anderson model. Our results show that the method successfully achieves uniform acceptance rates and reduces round-trip times across the temperature space. Furthermore, our proposed method offers a significant advantage over recently proposed policy gradient method that require careful hyperparameter tuning, while simultaneously preventing the constraint violations that destabilize optimization.