mlsanityData to 14 Oct 2026

Fractional Policy Gradients: Reinforcement Learning with Long-Term Memory

arXiv 2507.00073 ↗First posted 29 Jun 2025cs.LG stat.MLcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We propose Fractional Policy Gradients (FPG), a reinforcement learning framework incorporating fractional calculus for long-term temporal modeling in policy optimization. Standard policy gradient approaches face limitations from Markovian assumptions, exhibiting high variance and inefficient sampling. By reformulating gradients using Caputo fractional derivatives, FPG establishes power-law temporal correlations between state transitions. We develop an efficient recursive computation technique for fractional temporal-difference errors with constant time and memory requirements. Theoretical analysis shows FPG achieves asymptotic variance reduction of order O(t^(-alpha)) versus standard policy gradients while preserving convergence. Empirical validation demonstrates 35-68% sample efficiency gains and 24-52% variance reduction versus state-of-the-art baselines. This framework provides a mathematically grounded approach for leveraging long-range dependencies without computational overhead.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 18 references that are in the map. First posted on arXiv 29 Jun 2025.

Builds on

8 of 18 references.

References · 8

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  6. 2019Hindsight Credit Assignmentcited by 79
  7. 2023Fractional Deep Reinforcement Learning for Age-Minimal Mobile Edge Computingcited by 3
  8. 2019Autonomous Management of Energy-Harvesting IoT Nodes Using Deep Reinforcement Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State
  2. 2024Towards Provable Log Density Policy Gradient
  3. 2026Policy Gradient Methods for Non-Markovian Reinforcement Learning
  4. 2024The Definitive Guide to Policy Gradients in Deep Reinforcement Learning: Theory, Algorithms and Implementations
  5. 2026Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement Learning
  6. 2024Variational Delayed Policy Optimization
  7. 2023Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space
  8. 2026Limiting-Kernel Q(λ): Bridging Short and Long Horizons