mlsanityData to 14 Oct 2026

Distributional Reinforcement Learning for Energy-Based Sequential Models

arXiv 1912.08517 ↗First posted 18 Dec 2019cs.LG stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

Global Autoregressive Models (GAMs) are a recent proposal [Parshakova et al., CoNLL 2019] for exploiting global properties of sequences for data-efficient learning of seq2seq models. In the first phase of training, an Energy-Based model (EBM) over sequences is derived. This EBM has high representational power, but is unnormalized and cannot be directly exploited for sampling. To address this issue [Parshakova et al., CoNLL 2019] proposes a distillation technique, which can only be applied under limited conditions. By relating this problem to Policy Gradient techniques in RL, but in a distributional rather than optimization perspective, we propose a general approach applicable to any sequential EBM. Its effectiveness is illustrated on GAM-based experiments.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 1; 2023: 5; 2024: 5; 2025: 3.

17 papers.

Sort:

By area:

  1. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 120
  2. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 90
  3. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 82
  4. 2021Controlling Conditional Language Models without Catastrophic Forgettingcited by 32
  5. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  6. 2021Energy-Based Models for Code Generation under Compilability Constraintscited by 10
  7. 2024Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Modelcited by 9
  8. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  9. 2024BRAIn: Bayesian Reward-conditioned Amortized Inference for natural language generation from feedbackcited by 7
  10. 2024Guaranteed Generation from Large Language Modelscited by 7
  11. 2023disco: a toolkit for Distributional Control of Generative Modelscited by 3
  12. 2025Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversitycited by 3
5 more
  1. 2021Sampling from Discrete Energy-Based Models with Quality/Efficiency Trade-offscited by 2
  2. 2024Aligning language models with human preferencescited by 1
  3. 2025PIPA: Preference Alignment as Prior-Informed Statistical Estimationcited by 1
  4. 2023Structure-Aware Path Inference for Neural Finite State Transducerscited by 0
  5. 2025Reducing the Probability of Undesirable Outputs in Language Models Using Probabilistic Inferencecited by 0