mlsanityData to 14 Oct 2026

Dual Advantage Fields

arXiv 2606.04188 ↗First posted 2 Jun 2026cs.LG cs.AI cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Offline goal-conditioned reinforcement learning requires both long-horizon reachability estimates and local action comparisons. Dual goal representations provide value fields that capture global goal reachability, but they do not directly specify which action should be preferred at a given state. We propose Dual Advantage Fields (DAF), a policy-extraction method that turns a bilinear dual value model into a local advantage signal. Under bilinear dual parameterization, the goal embedding is the gradient of the value field with respect to the state representation, which directly specifies which action should be preferred. DAF introduces an action-effect model that predicts the discounted direction of change in state representation space induced by actions and scores them by the similarity between this change and the goal direction. In the realizable case, this score equals the goal-conditioned Bellman advantage, yielding a standard local policy-improvement guarantee. On OGBench locomotion, manipulation, and puzzle tasks, DAF improves aggregate RLiable metrics and performs strongly in settings where locally correct actions differ from direct movement toward the final goal.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 34 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

24 of 34 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement Learning12 Dec
  2. 2025Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learning11 Nov
  3. 2025Dual Goal Representations8 Oct
  4. 2025Horizon Reduction Makes RL Scalable4 Jun
  5. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making24 Jun

Other references · 19

  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  7. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  8. 2020Critic Regularized Regressioncited by 275
  9. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  10. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  11. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  12. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
7 more
  1. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  2. 2023Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularizationcited by 116
  3. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 68
  4. 2025Offline Goal-conditioned Reinforcement Learning with Quasimetric Representationscited by 24
  5. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learningcited by 23
  6. 2019The problem with DDPG: understanding failures in deterministic environments with sparse rewardscited by 20
  7. 2022Bilinear value networkscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Goal-conditioned Offline Reinforcement Learning through State Space Partitioning
  2. 2026Action-Sufficient Goal Representations
  3. 2026MSPR: Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learning
  4. 2026DAGR: State-Conditioned Goal Representations via Difference-Aware Goal Cross-Attention
  5. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression
  6. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learning
  7. 2026Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?
  8. 2026Beyond Success Rates: Trainability and Extractability for Offline GCRL