mlsanityData to 7 Oct 2026

LLM alignment / RLHF / preference optimisation

10 directions · 5,228 papers · 27 this week

Busiest this week

Most new since Mon 5 Oct

Safety alignment and jailbreak attacks

687 papers7 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

  1. How Narrative Wrapping Affects LLM Refusal: A Cross-Language Benchmark and Defense
  2. Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models
  3. SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing
  4. SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models
  5. The Model Plants the Trigger: Answer-Side Backdoor Attacks in Multi-Turn Large Language Models
Open →

All directions

Largest first · week from Mon 5 Oct

  1. LLM preference optimization methodspreference, alignment, language, optimizationMedium confidence in this grouping: mostly one topic, with blurry edges · About1,1323Medium confidence in this grouping: mostly one topic, with blurry edges
  2. Reward modeling and RLHFreward, human, feedback, rlhfMedium confidence in this grouping: mostly one topic, with blurry edges · About1,0292Medium confidence in this grouping: mostly one topic, with blurry edges
  3. Human values and AI feedback alignmentalignment, human, llms, languageLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · About7092Low confidence in this grouping: the papers may belong to more than one topic, or the group may change
  4. Diffusion and multimodal model alignmentpreference, reward, diffusion, generationMedium confidence in this grouping: mostly one topic, with blurry edges · About7092Medium confidence in this grouping: mostly one topic, with blurry edges
  5. Safety alignment and jailbreak attackssafety, alignment, safety alignment, harmfulHigh confidence in this grouping: the papers form one clear topic · About6877High confidence in this grouping: the papers form one clear topic
  6. Process reward models for reasoningreasoning, reward, process, trainingMedium confidence in this grouping: mostly one topic, with blurry edges · About5764Medium confidence in this grouping: mostly one topic, with blurry edges
  7. Preference alignment for recommenders and RAGpreference, recommendation, user, optimizationLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · About1690Low confidence in this grouping: the papers may belong to more than one topic, or the group may change
  8. Medical LLM alignmentmedical, clinical, preference, optimizationHigh confidence in this grouping: the papers form one clear topic · About1042High confidence in this grouping: the papers form one clear topic
  9. Reward hacking and misspecificationhacking, reward, reward hacking, rubricMedium confidence in this grouping: mostly one topic, with blurry edges · About942Medium confidence in this grouping: mostly one topic, with blurry edges
  10. Preference optimization for molecular designprotein, molecular, optimization, protein languageHigh confidence in this grouping: the papers form one clear topic · About190High confidence in this grouping: the papers form one clear topic
3 new this week aren’t in a direction yet
  1. Perceptually Aligned Evaluation of Style Transfer
  2. Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models
  3. Bradley-Terry model under general comparison graphs

Weekly activity

Papers per week, 22 Jun – 5 Oct (last week partial)

  1. DirectionConfidencepapers16 wks
  2. Safety alignment and jailbreak attacksHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 6; 29 Jun: 3; 6 Jul: 3; 13 Jul: 4; 20 Jul: 4; 27 Jul: 5; 3 Aug: 4; 10 Aug: 9; 17 Aug: 10; 24 Aug: 7; 31 Aug: 14; 7 Sep: 5; 14 Sep: 4; 21 Sep: 3; 28 Sep: 20; 5 Oct: 7687108
  3. LLM preference optimization methodsMedium confidence in this grouping: mostly one topic, with blurry edges · AboutMedium confidence in this grouping: mostly one topic, with blurry edgesPer week: 22 Jun: 4; 29 Jun: 10; 6 Jul: 6; 13 Jul: 3; 20 Jul: 5; 27 Jul: 6; 3 Aug: 2; 10 Aug: 5; 17 Aug: 1; 24 Aug: 9; 31 Aug: 9; 7 Sep: 6; 14 Sep: 5; 21 Sep: 5; 28 Sep: 12; 5 Oct: 31,13291
  4. Reward modeling and RLHFMedium confidence in this grouping: mostly one topic, with blurry edges · AboutMedium confidence in this grouping: mostly one topic, with blurry edgesPer week: 22 Jun: 8; 29 Jun: 7; 6 Jul: 3; 13 Jul: 0; 20 Jul: 2; 27 Jul: 9; 3 Aug: 8; 10 Aug: 5; 17 Aug: 3; 24 Aug: 1; 31 Aug: 6; 7 Sep: 7; 14 Sep: 3; 21 Sep: 10; 28 Sep: 8; 5 Oct: 21,02982
  5. Reward hacking and misspecificationMedium confidence in this grouping: mostly one topic, with blurry edges · AboutMedium confidence in this grouping: mostly one topic, with blurry edgesPer week: 22 Jun: 3; 29 Jun: 1; 6 Jul: 2; 13 Jul: 0; 20 Jul: 0; 27 Jul: 0; 3 Aug: 0; 10 Aug: 2; 17 Aug: 2; 24 Aug: 3; 31 Aug: 4; 7 Sep: 0; 14 Sep: 3; 21 Sep: 3; 28 Sep: 14; 5 Oct: 29439
  6. Human values and AI feedback alignmentLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · AboutLow confidence in this grouping: the papers may belong to more than one topic, or the group may changePer week: 22 Jun: 2; 29 Jun: 3; 6 Jul: 3; 13 Jul: 6; 20 Jul: 9; 27 Jul: 3; 3 Aug: 9; 10 Aug: 6; 17 Aug: 4; 24 Aug: 7; 31 Aug: 10; 7 Sep: 3; 14 Sep: 4; 21 Sep: 5; 28 Sep: 10; 5 Oct: 270986
  7. Diffusion and multimodal model alignmentMedium confidence in this grouping: mostly one topic, with blurry edges · AboutMedium confidence in this grouping: mostly one topic, with blurry edgesPer week: 22 Jun: 5; 29 Jun: 10; 6 Jul: 4; 13 Jul: 4; 20 Jul: 0; 27 Jul: 3; 3 Aug: 5; 10 Aug: 6; 17 Aug: 5; 24 Aug: 3; 31 Aug: 11; 7 Sep: 1; 14 Sep: 1; 21 Sep: 2; 28 Sep: 15; 5 Oct: 270977
  8. Process reward models for reasoningMedium confidence in this grouping: mostly one topic, with blurry edges · AboutMedium confidence in this grouping: mostly one topic, with blurry edgesPer week: 22 Jun: 9; 29 Jun: 3; 6 Jul: 5; 13 Jul: 5; 20 Jul: 3; 27 Jul: 4; 3 Aug: 3; 10 Aug: 3; 17 Aug: 2; 24 Aug: 8; 31 Aug: 6; 7 Sep: 2; 14 Sep: 2; 21 Sep: 4; 28 Sep: 7; 5 Oct: 457670
  9. Preference alignment for recommenders and RAGLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · AboutLow confidence in this grouping: the papers may belong to more than one topic, or the group may changePer week: 22 Jun: 3; 29 Jun: 1; 6 Jul: 0; 13 Jul: 1; 20 Jul: 0; 27 Jul: 1; 3 Aug: 1; 10 Aug: 5; 17 Aug: 1; 24 Aug: 1; 31 Aug: 3; 7 Sep: 1; 14 Sep: 1; 21 Sep: 2; 28 Sep: 2; 5 Oct: 016923
  10. Medical LLM alignmentHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 0; 29 Jun: 1; 6 Jul: 1; 13 Jul: 2; 20 Jul: 1; 27 Jul: 0; 3 Aug: 2; 10 Aug: 0; 17 Aug: 1; 24 Aug: 3; 31 Aug: 0; 7 Sep: 0; 14 Sep: 2; 21 Sep: 1; 28 Sep: 0; 5 Oct: 210416
  11. Preference optimization for molecular designHigh confidence in this grouping: the papers form one clear topic · AboutHigh confidence in this grouping: the papers form one clear topicPer week: 22 Jun: 0; 29 Jun: 1; 6 Jul: 0; 13 Jul: 0; 20 Jul: 0; 27 Jul: 0; 3 Aug: 0; 10 Aug: 0; 17 Aug: 0; 24 Aug: 0; 31 Aug: 0; 7 Sep: 0; 14 Sep: 0; 21 Sep: 0; 28 Sep: 0; 5 Oct: 0191