# A Two-fold Randomization Framework for Impulse Control Problems

arXiv: https://arxiv.org/abs/2509.12018 · HTML: https://mlsanity.com/paper/2509.12018/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2025.

## Built on by (most cited first, 3 of 3)

- [Continuous-time reinforcement learning for optimal switching over multiple regimes](https://mlsanity.com/paper/2512.04697/) (2025)
- [Deterministic Policy Gradient for Learning Equilibrium in Time-Inconsistent Control Problems](https://mlsanity.com/paper/2606.11798/) (2026)
- [Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence](https://mlsanity.com/paper/2609.14972/) (2026)

## Abstract (arXiv)

We propose and analyze a randomization scheme for a general class of impulse control problems. The solution to this randomized problem is characterized as the fixed point of a compound operator which consists of a regularized nonlocal operator and a regularized stopping operator. This approach allows us to derive a semi-linear Hamilton-Jacobi-Bellman (HJB) equation. Through an equivalent randomization scheme with a Poisson compound measure, we establish a verification theorem that implies the uniqueness of the solution. Via an iterative approach, we prove the existence of the solution. The existence-and-uniqueness result ensures the randomized problem is well-defined. We then demonstrate that our randomized impulse control problem converges to its classical counterpart as the randomization parameter pmb λ vanishes. This convergence, combined with the value function's C2,αloc regularity, confirms our framework provides a robust approximation and a foundation for developing learning algorithms. Under this framework, we propose an offline reinforcement learning (RL) algorithm. Its policy improvement step is naturally derived from the iterative approach from the existence proof, which enjoys a geometric convergence rate. We implement a model-free version of the algorithm and numerically demonstrate its effectiveness using a widely-studied example. The results show that our RL algorithm can learn the randomized solution, which accurately approximates its classical counterpart. A sensitivity analysis with respect to the volatility parameter σ in the state process effectively demonstrates the exploration-exploitation tradeoff.