mlsanityData to 7 Oct 2026

Score-Based Density Estimation from Pairwise Comparisons

arXiv 2510.09146 ↗First posted 10 Oct 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study density estimation from pairwise comparisons, motivated by expert knowledge elicitation and learning from human feedback. We relate the unobserved target density to a tempered winner density (marginal density of preferred choices), learning the winner's score via score-matching. This allows estimating the target by `de-tempering' the estimated winner density's score. We prove that the score vectors of the belief and the winner density are collinear, linked by a position-dependent tempering field. We give analytical formulas for this field and propose an estimator for it under the Bradley-Terry model. Using a diffusion model trained on tempered samples generated via score-scaled annealed Langevin dynamics, we can learn complex multivariate belief densities of simulated experts, from only hundreds to thousands of pairwise comparisons.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 55 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

32 of 55 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2022Flow Matching for Generative Modelingcited by 5,426
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2018Neural Ordinary Differential Equationscited by 3,733
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  12. 2015Variational Inference with Normalizing Flowscited by 2,161
20 more
  1. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  2. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  3. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  4. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  5. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 286
  6. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Modelscited by 131
  7. 2024Approaching Human-Level Forecasting with Language Modelscited by 102
  8. 2024All-in-one simulation-based inferencecited by 71
  9. 2023Turning large language models into cognitive modelscited by 66
  10. 2023Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEscited by 54
  11. 2024LLM Processes: Numerical Predictive Distributions Conditioned on Natural Languagecited by 54
  12. 2021Resampling Base Distributions of Normalizing Flowscited by 42
  13. 2024Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structurecited by 38
  14. 2023Detecting and Mitigating Mode-Collapse for Flow-based Sampling of Lattice Field Theoriescited by 25
  15. 2020Variational Bayesian Monte Carlo with Noisy Likelihoodscited by 23
  16. 2021Prior knowledge elicitation: The past, present, and futurecited by 23
  17. 2023A density estimation perspective on learning from pairwise human preferencescited by 23
  18. 2024AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modellingcited by 15
  19. 2024Preferential Normalizing Flowscited by 1
  20. 2024Expert-elicitation method for non-parametric joint priors using normalizing flowscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Bayesian Inference from Noisy Pairwise Comparisons
  2. 2024The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications
  3. 2026Learning a Ranking from Human Feedback in Log-Concave Random Utility Models
  4. 2024Operator-Informed Score Matching for Markov Diffusion Models
  5. 2024From optimal score matching to optimal sampling
  6. 2025From Score Matching to Diffusion: A Fine-Grained Error Analysis in the Gaussian Setting
  7. 2025DDPM Score Matching and Distribution Learning
  8. 2026Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights