# PEARL: Preconditioner Enhancement through Actor-critic Reinforcement Learning

arXiv: https://arxiv.org/abs/2501.10750 · HTML: https://mlsanity.com/paper/2501.10750/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 18 Jan 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 11 of 11)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Learning Algebraic Multigrid Using Graph Neural Networks](https://mlsanity.com/paper/2003.05744/) (2020)
- [Deep Reinforcement Learning with Enhanced Safety for Autonomous Highway Driving](https://mlsanity.com/paper/1910.12905/) (2019)
- [Deep Learning of Preconditioners for Conjugate Gradient Solvers in Urban Water Related Problems](https://mlsanity.com/paper/1906.06925/) (2019)
- [Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems](https://mlsanity.com/paper/2406.00809/) (2024)
- [Learning incomplete factorization preconditioners for GMRES](https://mlsanity.com/paper/2409.08262/) (2024)
- [Optimizing Sparse Matrix Multiplications for Graph Neural Networks](https://mlsanity.com/paper/2111.00352/) (2021)
- [A note on stabilizing reinforcement learning](https://mlsanity.com/paper/2111.12316/) (2021)
- [Deep Learning for Koopman Operator Estimation in Idealized Atmospheric Dynamics](https://mlsanity.com/paper/2409.06522/) (2024)
- [Concurrent Learning of Policy and Unknown Safety Constraints in Reinforcement Learning](https://mlsanity.com/paper/2402.15893/) (2024)

## Built on by (most cited first, 3 of 3)

- [Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown](https://mlsanity.com/paper/2507.15290/) (2025)
- [Acceleration of an algebraic multigrid pressure solver using graph neural networks](https://mlsanity.com/paper/2606.19251/) (2026)
- [An overview of machine learning-enhanced iterative methods for systems of linear and nonlinear equations](https://mlsanity.com/paper/2610.07211/) (2026)

## Abstract (arXiv)

We present PEARL (Preconditioner Enhancement through Actor-critic Reinforcement Learning), a novel approach to learning matrix preconditioners. Existing preconditioners such as Jacobi, Incomplete LU, and Algebraic Multigrid methods offer problem-specific advantages but rely heavily on hyperparameter tuning. Recent advances have explored using deep neural networks to learn preconditioners, though challenges such as misbehaved objective functions and costly training procedures remain. PEARL introduces a reinforcement learning approach for learning preconditioners, specifically, a contextual bandit formulation. The framework utilizes an actor-critic model, where the actor generates the incomplete Cholesky decomposition of preconditioners, and the critic evaluates them based on reward-specific feedback. To further guide the training, we design a dual-objective function, combining updates from the critic and condition number. PEARL contributes a generalizable preconditioner learning method, dynamic sparsity exploration, and cosine schedulers for improved stability and exploratory power. We compare our approach to traditional and neural preconditioners, demonstrating improved flexibility and iterative solving speed.