mlsanityData to 14 Oct 2026

The Role of Generator Access in Autoregressive Post-Training

arXiv 2604.04855 ↗First posted 6 Apr 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study how generator access constrains autoregressive post-training. The central question is whether the learner is confined to fresh root-start rollouts or can return to previously built prefixes and query the next-token rule there. In the root-start regime, output sampling, generated-token log probabilities, top-k reports, and full next-token distributions along sampled trajectories all reduce to one canonical experiment, limited by the on-policy probability of reaching informative prefixes. Weak prefix control breaks this barrier, and once control is available, richer observations such as conditional sampling or logits can outperform top-1 access. Changing only the generator interface creates an exponential gap for KL-regularized outcome-reward post-training.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 20 references that are in the map. First posted on arXiv 6 Apr 2026.

Builds on

13 of 20 references.

References · 13

  1. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 104
  2. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 97
  3. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 89
  4. 2025Is a Good Foundation Necessary for Efficient Reinforcement Learning? The Computational Role of the Base Model in Explorationcited by 36
  5. 2025The Coverage Principle: How Pre-Training Enables Post-Trainingcited by 30
  6. 2025RL for Reasoning by Adaptively Revealing Rationalescited by 16
  7. 2025Representation-Based Exploration for Language Models: From Test-Time to Post-Trainingcited by 16
  8. 2024The Power of Resets in Online Reinforcement Learningcited by 14
  9. 2023Learning Hidden Markov Models Using Conditional Samplescited by 12
  10. 2025On the Query Complexity of Verifier-Assisted Language Generationcited by 12
  11. 2024Model Stealing for Any Low-Rank Language Modelcited by 9
  12. 2026Restoring Exploration after Post-Training: Latent Exploration Decoding for Large Reasoning Modelscited by 4
1 more
  1. 2026Post-Training with Policy Gradients: Optimality and the Base Model Barriercited by 3

Built on by

Not cited yet.