# Data-driven rules for multidimensional reflection problems

arXiv: https://arxiv.org/abs/2311.06639 · HTML: https://mlsanity.com/paper/2311.06639/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 57 references that are in the map.
- First posted on arXiv 11 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [On the Sample Complexity of the Linear Quadratic Regulator](https://mlsanity.com/paper/1710.01688/) (2017)
- [Model-based Reinforcement Learning and the Eluder Dimension](https://mlsanity.com/paper/1406.1853/) (2014)
- [Logarithmic Regret for Reinforcement Learning with Linear Function Approximation](https://mlsanity.com/paper/2011.11566/) (2020)
- [Making Deep Q-learning methods robust to time discretization](https://mlsanity.com/paper/1901.09732/) (2019)
- [Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon](https://mlsanity.com/paper/2006.15316/) (2020)
- [Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controls](https://mlsanity.com/paper/2104.09311/) (2021)
- [Learning to reflect: A unifying approach for data-driven stochastic control strategies](https://mlsanity.com/paper/2104.11496/) (2021)
- [Statistical Learning with Sublinear Regret of Propagator Models](https://mlsanity.com/paper/2301.05157/) (2023)
- [A solvable two-dimensional singular stochastic control problem with non convex costs](https://mlsanity.com/paper/1411.2428/) (2014)
- [A Class of Solvable Stationary Singular Stochastic Control Problems](https://mlsanity.com/paper/1803.03464/) (2018)
- [Multidimensional singular control and related Skorokhod problem: sufficient conditions for the characterization of optimal controls](https://mlsanity.com/paper/2103.08487/) (2021)
- [On the Singular Control of Exchange Rates](https://mlsanity.com/paper/1712.02164/) (2017)
- [A Singular Stochastic Control Problem with Interconnected Dynamics](https://mlsanity.com/paper/1909.12045/) (2019)
- [Two sided ergodic singular control and mean field game for diffusions](https://mlsanity.com/paper/2306.09263/) (2023)

## Built on by (most cited first, 5 of 5)

- [Exploratory Optimal Stopping: A Singular Control Formulation](https://mlsanity.com/paper/2408.09335/) (2024)
- [Data-driven optimal stopping: A pure exploration analysis](https://mlsanity.com/paper/2312.05880/) (2023)
- [Learning to steer with Brownian noise](https://mlsanity.com/paper/2410.03221/) (2024)
- [Reinforcement Learning in Real Option Models](https://mlsanity.com/paper/2602.15643/) (2026)
- [Learning Optimal Search Strategies](https://mlsanity.com/paper/2603.02356/) (2026)

## Abstract (arXiv)

Over the recent past data-driven algorithms for solving stochastic optimal control problems in face of model uncertainty have become an increasingly active area of research. However, for singular controls and underlying diffusion dynamics the analysis has so far been restricted to the scalar case. In this paper we fill this gap by studying a multivariate singular control problem for reversible diffusions with controls of reflection type. Our contributions are threefold. We first explicitly determine the long-run average costs as a domain-dependent functional, showing that the control problem can be equivalently characterized as a shape optimization problem. For given diffusion dynamics, assuming the optimal domain to be strongly star-shaped, we then propose a gradient descent algorithm based on polytope approximations to numerically determine a cost-minimizing domain. Finally, we investigate data-driven solutions when the diffusion dynamics are unknown to the controller. Using techniques from nonparametric statistics for stochastic processes, we construct an optimal domain estimator, whose static regret is bounded by the minimax optimal estimation rate of the unreflected process' invariant density. In the most challenging situation, when the dynamics must be learned simultaneously to controlling the process, we develop an episodic learning algorithm to overcome the emerging exploration-exploitation dilemma and show that given the static regret as a baseline, the loss in its sublinear regret per time unit is of natural order compared to the one-dimensional case.