mlsanityData to 7 Oct 2026

Preference alignment for recommenders and RAG

169 papers0 this weekLow confidence in this grouping: the papers may belong to more than one topic, or the group may changeLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · About

Keywords: preference, recommendation, user, optimization, generation

Key facts. Preference alignment for recommenders and RAG: 169 papers on arXiv since 2021 (data through 7 Oct 2026). 15 of its 169 mapped papers (9%) cite “SpeechAlign: Aligning Speech Generation to Human Preferences” (2024, arXiv 2404.05600). 10 of its 169 mapped papers (6%) cite “MusicRL: Aligning Music Generation to Human Preferences” (2024, arXiv 2402.04229). 5 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

5 new papers, 17 Sep – 7 Oct; 0 this week.

  1. Earlier · 5
  2. SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences also fits: Medical LLM alignment
  3. PlaylistEval: Can Video-Language Judges Be Trusted at Day Scale and Beyond? also fits: Human values and AI feedback alignment
  4. Faithful Activation Verbalization: Reducing Hallucinations in LLM Representation Interpretation also fits: LLM preference optimization methods
  5. ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker
  6. Beyond Reference-Based Evaluation: Reward Models for Meta-Evaluation of Grammatical Error Correction also fits: Human values and AI feedback alignment

Foundations

Most cited by papers here

  1. SpeechAlign: Aligning Speech Generation to Human Preferences2024cited by 15 here
  2. MusicRL: Aligning Music Generation to Human Preferences2024cited by 10 here
  3. On Softmax Direct Preference Optimization for Recommendation2024cited by 8 here
  4. BATON: Aligning Text-to-Audio Model with Human Preference Feedback2024cited by 7 here
  5. Preference Alignment Improves Language Model-Based TTS2024cited by 7 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2017Deep reinforcement learning from human preferencescited by 24 here
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 16 here
  3. 2020Learning to summarize from human feedbackcited by 17 here
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 5 here
  5. 2022Training language models to follow instructions with human feedbackcited by 71 here
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 16 here
  7. 2022Scaling Laws for Reward Model Overoptimizationcited by 7 here
  8. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 5 here
  9. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 19 here
  10. 2024Preference Alignment Improves Language Model-Based TTScited by 7 here
14 more it builds on, not in this chain
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 8 here
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 5 here
  3. 2023Aligning Large Language Models through Synthetic Feedbackcited by 5 here
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 97 here
  5. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 6 here
  6. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 8 here
  7. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 6 here
  8. 2024Self-Rewarding Language Modelscited by 8 here
  9. 2024BATON: Aligning Text-to-Audio Model with Human Preference Feedbackcited by 7 here
  10. 2024MusicRL: Aligning Music Generation to Human Preferencescited by 10 here
  11. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 5 here
  12. 2024SpeechAlign: Aligning Speech Generation to Human Preferencescited by 15 here
  13. 2024On Softmax Direct Preference Optimization for Recommendationcited by 8 here
  14. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 6 here

The 24 most cited of 243.

Browse

30 of 169, ranked 6–35

  1. 6SpeechJudge: Towards Human-Level Judgment for Speech Naturalness20255
  2. 7Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance20254
  3. 8Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization20244
  4. 9KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models20244
  5. 10CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction20263
  6. 11Aligning Text-to-Music Evaluation with Human Preferences20253
  7. 12WavReward: Spoken Dialogue Models With Generalist Reward Evaluators20253
  8. 13LeVo: High-Quality Song Generation with Multi-Preference Alignment20253
  9. 14Aligning Spoken Dialogue Models from User Interactions20253
  10. 15Ask Optimal Questions: Aligning Large Language Models with Retriever's Preference in Conversation20243
All 30
  1. 16Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback20243
  2. 17Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation20243
  3. 18TuneJury: An Open Metric for Improving Music Generation Preference Alignment20262
  4. 19Differentiable Reward Optimization for LLM based TTS system20252
  5. 20No Verifiable Reward for Prosody: Toward Preference-Guided Prosody Learning in TTS20252
  6. 21From Scores to Preferences: Redefining MOS Benchmarking for Speech Quality Reward Modeling20252
  7. 22Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers20242
  8. 23Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback20242
  9. 24Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization20242
  10. 25TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization20242
  11. 26Training Dialogue Systems by AI Feedback for Improving Overall Dialogue Impression20251
  12. 27RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization20251
  13. 28Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward20251
  14. 29From Prompting to Alignment: A Generative Framework for Query Recommendation20251
  15. 30Aligning Generative Speech Enhancement with Perceptual Feedback20251
  16. 31Multi-Metric Preference Alignment for Generative Speech Restoration20251
  17. 32Aligning Large Language Models for Controllable Recommendations20241
  18. 33Reward-RAG: Enhancing RAG with Reward Driven Supervision20241
  19. 34RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment20241
  20. 35AWARE-US: Preference-Aware Infeasibility Resolution in Tool-Calling Agents20260