mlsanityData to 14 Oct 2026

PrT-Flow: Probability-Tractable Flow Matching for Reinforcement Learning

arXiv 2610.17667 ↗First posted 13 Oct 2026cs.RO cs.AIcited by 0

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Diffusion models and Flow Matching can represent flexible distributions over robot actions, but many reinforcement learning algorithms also require the probability of each action. We introduce PrT-Flow (Probability-Tractable Flow Matching), which computes probability density while retaining multimodal, multi-step action generation. Its Jacobian-Tractable Transformer (JTT) makes each sampling step invertible and enables efficient computation of its Jacobian determinant. We train these finite steps with Conditional Euler-step Training (CET), which supervises their displacement over one step during sampling rather than their instantaneous velocity. We prove that, under certain assumptions, CET has the same minimizers as fitting the Euler-solver updates of Flow Matching. PrT-Flow achieves 96% success on Robomimic Lift after 300 epochs of imitation learning, matching the diffusion policy reference, and 96% success on OGBench Cube-Triple after 600k online steps. These results demonstrate that invertible networks with tractable probability densities can learn effective robot policies and support SAC-style actor-critic training. Our code is available at https://github.com/Sunlight-zero/PrT-Flow .

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 27 of its 29 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

27 of 29 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Reinforcement Learning for Flow-Matching Policies with Density Transport7 Jun
  2. 2025Max-Entropy Reinforcement Learning with Flow Matching and A Case Study on LQR29 Dec
  3. 2025Flow Matching Policy Gradients28 Jul
  4. 2025Flow Q-Learning4 Feb
  5. 2024Learning Robotic Manipulation Policies from Point Clouds with Conditional Flow Matching11 Sep
  6. 2024Diffusion Policy Policy Optimization1 Sep
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2018Neural Ordinary Differential Equationscited by 3,779
  5. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  6. 2016Density estimation using Real NVPcited by 2,042
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  8. 2019Neural Spline Flowscited by 596
  9. 2024Flow Matching Guide and Codecited by 301
  10. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  11. 2024Normalizing Flows are Capable Generative Modelscited by 91
  12. 2019Leveraging exploration in off-policy algorithms via normalizing flowscited by 46
8 more
  1. 2019Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policiescited by 38
  2. 2020Sliced Iterative Normalizing Flowscited by 38
  3. 2025SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modelingcited by 31
  4. 2025FALCON: Few-step Accurate Likelihoods for Continuous Flowscited by 13
  5. 2025Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generatorscited by 8
  6. 2026SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flowscited by 3
  7. 2026GenPO++: Generative Policy Optimization with Jacobian-free Likelihood Ratioscited by 3
  8. 2026NFPO: Stabilized Policy Optimization of Normalizing Flow for Robotic Policy Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning
  2. 2026BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling
  3. 2025Streaming Flow Policy: Simplifying diffusion/flow-matching policies by treating action trajectories as flow trajectories
  4. 2025Extremum Flow Matching for Offline Goal Conditioned Reinforcement Learning
  5. 2023A Coupled Flow Approach to Imitation Learning
  6. 2025Reinforcement Learning for Flow-Matching Policies
  7. 2025Dense-Jump Flow Matching with Non-Uniform Time Scheduling for Robotic Policies: Mitigating Multi-Step Inference Degradation
  8. 2026Flowing With Purpose: Latent Action Guided Flow Matching Policies For Robotic Manipulation