# Performance guarantees for model-based Approximate Dynamic Programming in continuous spaces

arXiv: https://arxiv.org/abs/1602.07273 · HTML: https://mlsanity.com/paper/1602.07273/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2016.

## Built on by (most cited first, 3 of 3)

- [Proximal Point Imitation Learning](https://mlsanity.com/paper/2209.10968/) (2022)
- [Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2112.14004/) (2021)
- [A Risk-Averse Preview-based Q-Learning Algorithm: Application to Highway Driving of Autonomous Vehicles](https://mlsanity.com/paper/2112.03232/) (2021)

## Abstract (arXiv)

We study both the value function and Q-function formulation of the Linear Programming approach to Approximate Dynamic Programming. The approach is model-based and optimizes over a restricted function space to approximate the value function or Q-function. Working in the discrete time, continuous space setting, we provide guarantees for the fitting error and online performance of the policy. In particular, the online performance guarantee is obtained by analyzing an iterated version of the greedy policy, and the fitting error guarantee by analyzing an iterated version of the Bellman inequality. These guarantees complement the existing bounds that appear in the literature. The Q-function formulation offers benefits, for example, in decentralized controller design, however it can lead to computationally demanding optimization problems. To alleviate this drawback, we provide a condition that simplifies the formulation, resulting in improved computational times.