mlsanityData to 7 Oct 2026

Learnable Chernoff Baselines for Inference-Time Alignment

arXiv 2602.07738 ↗First posted 8 Feb 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We study inference-time reward-guided alignment for generative models. Existing methods often rely on either architecture-specific adaptations or computationally costly inference procedures. We introduce Learnable Chernoff Baselines (LCBs) as a method for efficiently and approximately sampling from the exponentially tilted kernels that arise from KL-regularized reward alignment. Using only black-box sampling access to the pretrained model, LCBs implement a form of rejection sampling with adaptively selected acceptance probabilities, which allows fine-grained control over inference-compute scaling. We establish total-variation guarantees to the ideal aligned model, and demonstrate in both continuous and discrete diffusion settings that LCB sampling closely matches ideal rejection sampling while using substantially fewer queries to the pretrained model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 48 references that are in the map. First posted on arXiv 8 Feb 2026.

Builds on

39 of 48 references.

References · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2022Video Diffusion Modelscited by 2,280
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  10. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  11. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  12. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
27 more
  1. 2022Denoising Diffusion Restoration Modelscited by 951
  2. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  3. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  4. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  5. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  6. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  7. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  8. 2023Universal Guidance for Diffusion Modelscited by 426
  9. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  10. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 229
  11. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 218
  12. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  13. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  14. 2024BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Samplingcited by 120
  15. 2024Theoretical guarantees on the best-of-n alignment policycited by 116
  16. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 91
  17. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 82
  18. 2023Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Modelcited by 79
  19. 2024Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimizationcited by 78
  20. 2023Reward-Directed Conditional Diffusion: Provable Distribution Estimation and Reward Improvementcited by 49
  21. 2024Asymptotics of Language Model Alignmentcited by 46
  22. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 42
  23. 2025Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcingcited by 37
  24. 2024InfAlign: Inference-aware language model alignmentcited by 34
  25. 2025Psi-Sampler: Initial Particle Sampling for SMC-Based Inference-Time Reward Alignment in Score Modelscited by 15
  26. 2025CoDe: Blockwise Control for Denoising Diffusion Modelscited by 14
  27. 2024Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplerscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards a theory of inference-time alignment with unknown rewards
  2. 2026Diamond Maps: Efficient Reward Alignment via Stochastic Flow Maps
  3. 2025Inference-time Alignment in Continuous Space
  4. 2026Meta Flow Maps enable scalable reward alignment
  5. 2025Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner
  6. 2026Gradient-Guided Reward Optimization for Inference-time Alignment
  7. 2026The tractability landscape of diffusion alignment: regularization, rewards, and computational primitives
  8. 2026Revisiting the (Sub)Optimality of Best-of-N for Inference-Time Alignment