mlsanityData to 14 Oct 2026

Learning Distributions from Samples and Qualities

arXiv 2610.17956 ↗First posted 13 Oct 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study parameter estimation when each sample from a continuous parametric distribution is accompanied by a binary quality label governed by the same unknown parameter. By imposing hard feasibility constraints, deterministic labels can accelerate estimation from the classical n-1/2 rate of smooth quality-aware M-estimators to n-1. Combining PAC guarantees with local error-to-parameter conversions, we establish this rate for both identifiable and finitely ambiguous quality classifiers, with likelihood information resolving the latter ambiguity. When the quality model is known only up to an additive constant, we construct threshold-free pairwise preferences from oppositely labeled samples and retain the accelerated rate despite the unknown threshold. We further show that pointwise qualities and pairwise preferences are incomparable: either can be strictly more informative than the other. Our assumptions hold for Gaussian, Laplace, and exponential families under thresholded log-likelihood rewards. Finally, a minimax lower bound establishes the optimal n-1 dependence on sample size, up to logarithmic and dimension-dependent factors, while experiments confirm the benefits of constraint-based estimation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 67 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

24 of 67 references.

References · 24

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  8. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  9. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  11. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 399
  12. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247
12 more
  1. 2024Generalized Preference Optimization: A Unified Approach to Offline Alignmentcited by 157
  2. 2024MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Modelscited by 100
  3. 2015Simple, Robust and Optimal Ranking from Pairwise Comparisonscited by 70
  4. 2015Estimation from Pairwise Comparisons: Sharp Minimax Bounds with Topology Dependencecited by 61
  5. 2022JuMP 1.0: Recent improvements to a modeling language for mathematical optimizationcited by 37
  6. 2015Refined Error Bounds for Several Learning Algorithmscited by 25
  7. 2014Minimax-optimal Inference from Partial Rankingscited by 24
  8. 2023ULMA: Unified Language Model Alignment with Human Demonstration and Point-wise Preferencecited by 12
  9. 2024Learning Linear Utility Functions From Pairwise Comparison Queriescited by 8
  10. 2024Unified Preference Optimization: Language Model Alignment Beyond the Preference Frontiercited by 4
  11. 2025Leveraging Sparsity for Sample-Efficient Preference Learning: A Theoretical Perspectivecited by 3
  12. 2025Learning Parametric Distributions from Samples and Preferencescited by 1

Built on by

Not cited yet.