# Functional Natural Policy Gradients

arXiv: https://arxiv.org/abs/2603.28681 · HTML: https://mlsanity.com/paper/2603.28681/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 30 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 14 of 14)

- [Doubly Robust Off-policy Value Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1511.03722/) (2015)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes](https://mlsanity.com/paper/1908.08526/) (2019)
- [Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning](https://mlsanity.com/paper/1909.05850/) (2019)
- [Offline Multi-Action Policy Learning: Generalization and Optimization](https://mlsanity.com/paper/1810.04778/) (2018)
- [Fast Rates for Contextual Linear Optimization](https://mlsanity.com/paper/2011.03030/) (2020)
- [Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2202.09667/) (2022)
- [Applied Causal Inference Powered by ML and AI](https://mlsanity.com/paper/2403.02467/) (2024)
- [Semi-Parametric Efficient Policy Learning with Continuous Actions](https://mlsanity.com/paper/1905.10116/) (2019)
- [More Efficient Policy Learning via Optimal Retargeting](https://mlsanity.com/paper/1906.08611/) (2019)
- [Functional Bilevel Optimization for Machine Learning](https://mlsanity.com/paper/2403.20233/) (2024)
- [Efficient Policy Learning from Surrogate-Loss Classification Reductions](https://mlsanity.com/paper/2002.05153/) (2020)
- [Learning Theory for Kernel Bilevel Optimization](https://mlsanity.com/paper/2502.08457/) (2025)
- [Contextual Linear Optimization with Partial Feedback](https://mlsanity.com/paper/2405.16564/) (2024)

## Built on by (most cited first, 1 of 1)

- [Semiparametric Efficient Bilevel Gradient Estimation](https://mlsanity.com/paper/2605.21341/) (2026)

## Abstract (arXiv)

We propose a cross-fitted debiasing device for policy learning from offline data. A key consequence of the resulting learning principle is √ N regret even for policy classes with complexity greater than Donsker, provided a product-of-errors nuisance remainder is O(N-1/2). The regret bound factors into a plug-in policy error factor governed by policy-class complexity and an environment nuisance factor governed by the complexity of the environment dynamics, making explicit how one may be traded against the other.