mlsanityData to 7 Oct 2026

Partition Function Estimation under Bounded f-Divergence

arXiv 2602.23535 ↗First posted 26 Feb 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study the statistical complexity of estimating partition functions given sample access to a proposal distribution and an unnormalized density ratio for a target distribution. While partition function estimation is a classical problem, existing guarantees typically rely on structural assumptions about the domain or model geometry. We instead provide a general, information-theoretic characterization that depends only on the relationship between the proposal and target distributions. Our analysis introduces the integrated coverage profile, a functional that quantifies how much target mass lies in regions where the density ratio is large. We show that integrated coverage tightly characterizes the sample complexity of multiplicative partition function estimation and provide matching lower bounds. We further express these bounds in terms of f-divergences, yielding sharp phase transitions depending on the growth rate of f and recovering classical results as a special case while extending to heavy-tailed regimes. Matching lower bounds establish tightness in all regimes. As applications, we derive improved finite-sample guarantees for importance sampling and self-normalized importance sampling, and we show a strict separation between the complexity of approximate sampling and counting under the same divergence constraints. Our results unify and generalize prior analyses of importance sampling, rejection sampling, and heavy-tailed mean estimation, providing a minimal-assumption theory of partition function estimation. Along the way we introduce new technical tools including new connections between coverage and f-divergences as well as a generalization of the classical Paley-Zygmund inequality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 43 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

13 of 43 references.

References · 13

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2015Importance Weighted Autoencoderscited by 454
  3. 2014A* Samplingcited by 150
  4. 2019Mean estimation and regression under heavy-tailed distributions--a surveycited by 133
  5. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 102
  6. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 91
  7. 2025The Coverage Principle: How Pre-Training Enables Post-Trainingcited by 25
  8. 2025Accelerating RL for LLM Reasoning with Optimal Advantage Regressioncited by 23
  9. 2023The Sample Complexity of Approximate Rejection Sampling with Applications to Smoothed Online Learningcited by 13
  10. 2023Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Noise-Contrastive Estimation, and beyondcited by 13
  11. 2016A Faster Approximation Algorithm for the Gibbs Partition Functioncited by 4
  12. 2024Some Notes on the Sample Complexity of Approximate Channel Simulationcited by 4
1 more
  1. 2025Optimality in importance sampling: a gentle surveycited by 4

Built on by

Not cited yet.