# Expressivity and Statistical Trade-offs in Diffusion Policy Learning

arXiv: https://arxiv.org/abs/2607.07967 · HTML: https://mlsanity.com/paper/2607.07967/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 13 of its 68 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)
- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies](https://mlsanity.com/paper/2304.10573/) (2023)
- [3D Diffuser Actor: Policy Diffusion with 3D Scene Representations](https://mlsanity.com/paper/2402.10885/) (2024)
- [Policy Representation via Diffusion Probability Model for Reinforcement Learning](https://mlsanity.com/paper/2305.13122/) (2023)
- [Efficient Online Reinforcement Learning for Diffusion Policy](https://mlsanity.com/paper/2502.00361/) (2025)
- [DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets](https://mlsanity.com/paper/2404.19264/) (2024)
- [Covering Numbers for Deep ReLU Networks with Applications to Function Approximation and Nonparametric Regression](https://mlsanity.com/paper/2410.06378/) (2024)
- [Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching](https://mlsanity.com/paper/2606.22630/) (2026)
- [Reparameterization Flow Policy Optimization](https://mlsanity.com/paper/2602.03501/) (2026)
- [Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making](https://mlsanity.com/paper/2512.08280/) (2025)
- [GenPO++: Generative Policy Optimization with Jacobian-free Likelihood Ratios](https://mlsanity.com/paper/2606.06967/) (2026)
- [Truncated Rectified Flow Policy for Reinforcement Learning with One-Step Sampling](https://mlsanity.com/paper/2604.09159/) (2026)
- [Score-Based One-step MeanFlow Policy Optimization](https://mlsanity.com/paper/2605.23365/) (2026)

## Abstract (arXiv)

Diffusion-based policies have recently emerged as powerful policy parameterizations for reinforcement learning, representing state-conditioned action distributions as terminal laws of diffusion processes with parameterized drifts. This terminal-law representation has shown substantial expressive flexibility in practice, enabling diffusion policies to model complex, multimodal, and highly non-Gaussian action distributions; however, it remains unclear what mathematically drives this expressivity and how to fully exploit it when the policy is learned from finite data. In this paper, we identify the drift Lipschitz budget K as a central quantity governing the expressivity and statistical behavior of diffusion policies. We quantify expressivity through approximation: diffusion policies with K-Lipschitz drifts can concentrate near optimal deterministic policies and achieve value approximation error of order 1/K; moreover, we prove a matching lower bound under nondegenerate diffusion noise. This increased expressivity comes with a statistical cost. When the drift is parameterized by neural networks, increasing K improves approximation but increases statistical complexity. Balancing these two terms yields a finite-sample performance gap of order tildeO(n-2/(m+6)) for generic neural-network drifts, and a sharper rate tildeO(n-2/(m+4)) for one-sided dissipative drift classes, where n is the sample size and m is the dimension of the state space. Numerical experiments provide empirical evidence for the sample-dependent trade-off in K, supporting both theoretical regimes. Our framework also suggests a practical implementation principle: choose the diffusion budget K according to the available sample size, and then select a neural-network architecture with the corresponding fixed Lipschitz coefficient.