mlsanityData to 14 Oct 2026

Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control

arXiv 2610.11619 ↗First posted 8 Oct 2026math.OC cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We develop a model-free policy gradient method for discrete-time mean-field control (MFC). In MFC, the policy affects the objective both through the controlled dynamics and through the population distribution. Standard REINFORCE estimators capture the first effect but not the second. We introduce Transport REINFORCE, a transport map-based approach that perturbs a suitable transformation of the population distribution to estimate this missing mean-field contribution. The method applies to both finite and continuous state spaces. In finite state spaces, we perturb the population distribution directly on the probability simplex through a convex combination of the current population weights and random weights. In continuous state spaces, we project the population distribution onto the manifold of Gaussian mixtures, and then randomize it via a transport map that ensures the perturbed law remains within this manifold. We prove consistency of the perturbed objective and gradient as the perturbation vanishes, and derive bias and mean-square error bounds for the resulting sample-based gradient estimator. Numerical experiments on several MFC benchmarks show that Transport REINFORCE improves over standard REINFORCE.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 28 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

9 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficients23 Jun

Other references · 8

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methodscited by 28
  3. 2024Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noisecited by 6
  4. 2026Model-free policy gradient for discrete-time mean-field controlcited by 4
  5. 2026Policy Gradient for Continuous-Time Mean-Field Controlcited by 4
  6. 2026Learning operators on labelled conditional distributions with applications to mean field control of non exchangeable systemscited by 3
  7. 2025Probabilistic Analysis of Graphon Mean Field Controlcited by 2
  8. 2026Non-Exchangeable Mean Field Markov Decision Processes with common noise : from Bellman equation to quantitative propagation of chaoscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Data
  2. 2026Actor-Critic Learning for Extended Mean Field Control with Deterministic Policies
  3. 2025On the Contractivity of Stochastic Interpolation Flow
  4. 2025Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference
  5. 2026The Fundamental Limits of Valid Transport Map Estimation
  6. 2026Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning
  7. 2026Mean-Field Path-Integral Diffusion: From Samples to Interacting Agents
  8. 2026Mean Field Reinforcement Learning