# Preference alignment for recommenders and RAG

Area: [LLM alignment / RLHF / preference optimisation](https://mlsanity.com/alignment/index.md). HTML: https://mlsanity.com/alignment/preference-alignment-for-recommenders-and-rag/

## Key facts

- Preference alignment for recommenders and RAG: 175 papers on arXiv since 2021 (data through 14 Oct 2026).
- 15 of its 175 mapped papers (9%) cite “SpeechAlign: Aligning Speech Generation to Human Preferences” (2024, arXiv 2404.05600).
- 10 of its 175 mapped papers (6%) cite “MusicRL: Aligning Music Generation to Human Preferences” (2024, arXiv 2402.04229).
- 9 new papers 17 Sep–14 Oct.

## Most cited here

- [SpeechAlign: Aligning Speech Generation to Human Preferences](https://mlsanity.com/paper/2404.05600/) (2024), cited by 15 papers here
- [MusicRL: Aligning Music Generation to Human Preferences](https://mlsanity.com/paper/2402.04229/) (2024), cited by 10 papers here
- [On Softmax Direct Preference Optimization for Recommendation](https://mlsanity.com/paper/2406.09215/) (2024), cited by 8 papers here
- [BATON: Aligning Text-to-Audio Model with Human Preference Feedback](https://mlsanity.com/paper/2402.00744/) (2024), cited by 7 papers here
- [Preference Alignment Improves Language Model-Based TTS](https://mlsanity.com/paper/2409.12403/) (2024), cited by 7 papers here
- [SpeechJudge: Towards Human-Level Judgment for Speech Naturalness](https://mlsanity.com/paper/2511.07931/) (2025), cited by 5 papers here
- [Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization](https://mlsanity.com/paper/2404.09956/) (2024), cited by 4 papers here
- [KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models](https://mlsanity.com/paper/2408.03297/) (2024), cited by 4 papers here
- [Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance](https://mlsanity.com/paper/2502.05236/) (2025), cited by 4 papers here
- [Ask Optimal Questions: Aligning Large Language Models with Retriever's Preference in Conversation](https://mlsanity.com/paper/2402.11827/) (2024), cited by 3 papers here

## New (17 Sep–14 Oct)

- [Benchmarking and Training Reward Models for Cross-Source Evidence Grounding in Search Agents](https://mlsanity.com/paper/2610.17827/) (2026-10-13)
- [WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search](https://mlsanity.com/paper/2610.16927/) (2026-10-12)
- [AlphaRLM: Aligning Language Models for Literature-Grounded Symbolic Factor Discovery](https://mlsanity.com/paper/2610.14959/) (2026-10-10)
- [SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences](https://mlsanity.com/paper/2609.34582/) (2026-09-28)
- [PlaylistEval: Can Video-Language Judges Be Trusted at Day Scale and Beyond?](https://mlsanity.com/paper/2609.34314/) (2026-09-28)
- [Faithful Activation Verbalization: Reducing Hallucinations in LLM Representation Interpretation](https://mlsanity.com/paper/2609.34033/) (2026-09-27)
- [ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker](https://mlsanity.com/paper/2609.31002/) (2026-09-25)
- [EmphTTS: an emphasis-control TTS with reinforcement learning](https://mlsanity.com/paper/2609.27599/) (2026-09-23)
- [Beyond Reference-Based Evaluation: Reward Models for Meta-Evaluation of Grammatical Error Correction](https://mlsanity.com/paper/2609.21231/) (2026-09-18)