mlsanityData to 7 Oct 2026

AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling

arXiv 2601.08097 ↗First posted 13 Jan 2026cs.CL cs.LGcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

Reward modeling is essential for aligning large language models with human preferences, yet predominant architectures rely on a static pooling strategy to condense sequences into scalar scores. This paradigm, however, suffers from two key limitations: a static inductive bias that misaligns with task-dependent preference signals, and a representational mismatch, as the backbone's optimization for generation leaves its representations ill-suited to fine-grained discrimination. To address this, we propose AdaJudge, a unified framework that jointly adapts representation and aggregation. AdaJudge first improves backbone representations into a discrimination-oriented space via gated refinement blocks. It then replaces the static readout with an adaptive multi-view pooling module, which dynamically routes and combines evidence. Extensive experiments on RM-Bench and JudgeBench show that AdaJudge outperforms strong off-the-shelf reward models and traditional pooling baselines.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 26 of its 40 references that are in the map. First posted on arXiv 13 Jan 2026.

Builds on

26 of 40 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Ask a Strong LLM Judge when Your Reward Model is Uncertain23 Oct
  2. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy2 Jul
  3. 2025HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages16 May
  4. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs24 Oct
  5. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style21 Oct
  6. 2024Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown1 Oct
  7. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts18 Jun
  8. 2024Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs14 Jun
  9. 2023Reward Model Ensembles Help Mitigate Overoptimization4 Oct
  10. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  11. 2019Fine-Tuning Language Models from Human Preferences18 Sep
  12. 2017Deep reinforcement learning from human preferences12 Jun

Other references · 14

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  10. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
  11. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
  12. 2019Depth-Adaptive Transformercited by 227
2 more
  1. 2013Deep Learning of Representations: Looking Forwardcited by 84
  2. 2025Robust Noise Attenuation via Adaptive Pooling of Transformer Outputscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivitycited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models
  2. 2025PIRA: Preference-Oriented Instruction-Tuned Reward Models with Dual Aggregation
  3. 2026Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization
  4. 2026One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Models
  5. 2024RewardBench: Evaluating Reward Models for Language Modeling
  6. 2026MARS: Margin and Semantic-Aware Data Augmentation for Reward Modeling
  7. 2026CAMEL: Confidence-Gated Reflection for Reward Modeling
  8. 2026RMGAP: Benchmarking the Generalization of Reward Models across Diverse Preferences