A Course in Dynamic Optimization
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
These lecture notes are derived from a graduate-level course in dynamic optimization, offering an introduction to techniques and models extensively used in management science, economics, operations research, engineering, and computer science. The course emphasizes the theoretical underpinnings of discrete-time dynamic programming models and advanced algorithmic strategies for solving these models. Unlike typical treatments, it provides a proof for the principle of optimality for upper semi-continuous dynamic programming, a middle ground between the simpler countable state space case citebertsekas2012dynamic, and the involved universally measurable case citebertsekas1996stochastic. This approach is sufficiently rigorous to include important examples such as dynamic pricing, consumption-savings, and inventory management models. The course also delves into the properties of value and policy functions, leveraging classical results citetopkis1998supermodularity and recent developments. Additionally, it offers an introduction to reinforcement learning, including a formal proof of the convergence of Q-learning algorithms. Furthermore, the notes delve into policy gradient methods for the average reward case, presenting a convergence result for the tabular case in this context. This result is simple and similar to the discounted case but appears to be new.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 2 of its 20 references that are in the map. First posted on arXiv 6 Aug 2024.
Builds on
2 of 20 references.
References · 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reinforcement Learning for Continuous-Time Jump Markov Decision Processes with Applications to Network Dynamic Pricing
- 2026A Survey of Reinforcement Learning For Economics
- 2024VC Theory for Inventory Policies
- 2026A Comparative Study of Dynamic Programming and Reinforcement Learning in Finite Horizon Dynamic Pricing
- 2026Benign Nonconvex Landscape for Policy Optimization: Infinite-Horizon Discounted MDPs with General State and Action Spaces
- 2024Structure Matters: Dynamic Policy Gradient
- 2026Relating Reinforcement Learning to Dynamic Programming-Based Planning
- 2025Efficient Algorithms for Mitigating Uncertainty and Risk in Reinforcement Learning