mlsanityData to 14 Oct 2026

Generative Modeling of Discrete Latent Structures via Dynamic Policy Gradients

arXiv 2606.07400 ↗First posted 5 Jun 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL with diffusion and flow models

Abstract (arXiv)

Many scientific problems require inferring unobserved mechanistic latent states from indirect observations. While classical approaches, including expectation maximization, do not scale to combinatorially large spaces, deep learning approaches such as variational autoencoders typically form artificial latent states rather than reconstructing the mechanistic ground-truth states. Here, we introduce GReinSS, a policy learning framework that uses dynamically rescaled rewards to learn latent state distributions that maximize the observed data likelihood. We show that GReinSS accurately reconstructs simulated latent sets and latent graphs, outperforming alternative policy learning and generative modeling baselines. Additionally, GReinSS reconstructs isoforms from real short-read RNA sequencing data that better match isoforms detected by orthogonal long-read sequencing than the standard RSEM algorithm. Overall, GReinSS is a principled and practically effective approach for generative modeling and inference of combinatorial latent states from indirect observations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 46 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

9 of 46 references.

References · 9

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  2. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  3. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 376
  4. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 251
  5. 2014Neural Variational Inference and Learning in Belief Networkscited by 117
  6. 2016Variational inference for Monte Carlo objectivescited by 84
  7. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applicationscited by 38
  8. 2020Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Networkcited by 27
  9. 2018Weight Thresholding on Complex Networkscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Maximum Likelihood Learning of Latent Dynamics Without Reconstruction
  2. 2026Learn from your own latents and not from tokens: A sample-complexity theory
  3. 2026Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning
  4. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
  5. 2025From Classical Probabilistic Latent Variable Models to Modern Generative AI: A Unified Perspective
  6. 2026f-Trajectory Balance: A Loss Family for Tuning GFlowNets, Generative Models, and LLMs with Off- and On-Policy Data
  7. 2025Distillation of a tractable model from the VQ-VAE
  8. 2026Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization