mlsanityData to 14 Oct 2026

RLHF: A comprehensive Survey for Cultural, Multimodal and Low Latency Alignment Methods

arXiv 2511.03939 ↗First posted 6 Nov 2025cs.LG cs.AI cs.CLcited by 2

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Reinforcement Learning from Human Feedback (RLHF) is the standard for aligning Large Language Models (LLMs), yet recent progress has moved beyond canonical text-based methods. This survey synthesizes the new frontier of alignment research by addressing critical gaps in multi-modal alignment, cultural fairness, and low-latency optimization. To systematically explore these domains, we first review foundational algo- rithms, including PPO, DPO, and GRPO, before presenting a detailed analysis of the latest innovations. By providing a comparative synthesis of these techniques and outlining open challenges, this work serves as an essential roadmap for researchers building more robust, efficient, and equitable AI systems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 9 of its 24 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

9 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 8

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2015Trust Region Policy Optimizationcited by 2,580
  7. 2021Ethical and social risks of harm from Language Modelscited by 966
  8. 2020Aligning AI With Shared Human Valuescited by 723

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Light Alignment Improves LLM Safety via Model Self-Reflection with a Single Neuroncited by 0
  2. 2026S2T-RLHF: Hierarchical Credit Assignment for Stable Preference-Based RLHFcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On The Global Convergence Of Online RLHF With Neural Parametrization
  2. 2025RLTHF: Targeted Human Feedback for LLM Alignment
  3. 2025Beyond RLHF: A Unified Theoretical Framework of Alignment
  4. 2024ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback
  5. 2024SAIL: Self-Improving Efficient Online Alignment of Large Language Models
  6. 2023CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignment
  7. 2024AI Alignment through Reinforcement Learning from Human Feedback? Contradictions and Limitations
  8. 2025Aligning Frozen LLMs by Reinforcement Learning: An Iterative Reweight-then-Optimize Approach