mlsanityData to 14 Oct 2026

DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning

arXiv 2610.16597 ↗First posted 12 Oct 2026cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL with diffusion and flow models also fits: RL for robot locomotion and manipulation

Abstract (arXiv)

Expressive flow policies can represent multiple high-value actions, but sampling them with few integration steps while retaining tractable action densities remains challenging. We introduce DEFlow, a deep equilibrium flow policy that replaces explicit Euler sampling with implicit Euler integration and solves all intermediate actions jointly as an equilibrium system. We combine implicit differentiation with a discrete change-of-variables formula for the final-action density, enabling off-policy training within soft actor-critic without backpropagating through solver iterations. DEFlow achieves an average relative improvement of approximately 10% in mean successful coverage over the strongest baseline across four Multi-Goal docking variants. It also achieves the highest mean time-averaged return on four of five manipulation tasks, while remaining broadly competitive across five MuJoCo tasks. These results support implicit flow policies for tasks requiring both diversity and precision, at the cost of additional nonlinear and linear solves. Primary

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 19 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

14 of 19 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Flow Q-Learning4 Feb
  2. 2025DIME:Diffusion-Based Maximum Entropy Reinforcement Learning4 Feb

Other references · 12

  1. 2022Flow Matching for Generative Modelingcited by 5,702
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Variational Inference with Normalizing Flowscited by 2,170
  4. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  7. 2019Deep Equilibrium Modelscited by 615
  8. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  9. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 138
  10. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 100
  11. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 72
  12. 2023Backpropagation through Back Substitution with a Backslashcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FlowDPG: Deterministic Policy Gradient on Flow Matching Policies for Real-World Manipulation
  2. 2025Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning
  3. 2026Truncated Rectified Flow Policy for Reinforcement Learning with One-Step Sampling
  4. 2025SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modeling
  5. 2025Max-Entropy Reinforcement Learning with Flow Matching and A Case Study on LQR
  6. 2025Flow-Based Single-Step Completion for Efficient and Expressive Policy Learning
  7. 2026RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation
  8. 2026Reinforcement Learning via Fixed-point Flow Matching