mlsanityData to 7 Oct 2026

JudgeLRM: Large Reasoning Models as a Judge

arXiv 2504.00050 ↗First posted 31 Mar 2025cs.CL cs.AIcited by 76

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models (LLMs) are increasingly adopted as evaluators, offering a scalable alternative to human annotation. However, existing supervised fine-tuning (SFT) approaches often fall short in domains that demand complex reasoning. Judgment is inherently reasoning-intensive: beyond surface-level scoring, it requires verifying evidence, identifying errors, and justifying decisions. Through the analysis of evaluation tasks, we find a negative correlation between SFT performance gains and the proportion of reasoning-demanding samples, revealing the limits of SFT in such scenarios. To address this, we introduce JudgeLRM, a family of judgment-oriented LLMs, trained using reinforcement learning (RL) with judge-wise, outcome-driven rewards to activate reasoning capabilities. JudgeLRM consistently outperform SFT-tuned baselines in the same size, as well as other RL and SFT variants, and even surpass state-of-the-art reasoning models: notably, JudgeLRM-3B/4B exceeds GPT-4, while JudgeLRM-7B/8B/14B outperforms DeepSeek-R1 by over 2% in F1 score, with particularly strong gains on reasoning-heavy tasks. Our findings underscore the value of RL in unlocking reasoning-aligned LLM judges.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 21 of its 28 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

21 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Language models show human-like content effects on reasoning tasks14 Jul

Other references · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2024A Survey on LLM-as-a-Judgecited by 1,384
  8. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  9. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  10. 2023Large Language Modelscited by 774
  11. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  12. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
8 more
  1. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  2. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  3. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 314
  4. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  5. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
  6. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  7. 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Modelscited by 144
  8. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 82

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 41; 2026: 35.

76 papers.

Sort:

By area:

  1. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  3. 2025RM-R1: Reward Modeling as Reasoningcited by 149
  4. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  5. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 118
  6. 2025J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learningcited by 73
  7. 2025Reward Reasoning Modelcited by 47
  8. 2025Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewardscited by 47
  9. 2026Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Trainingcited by 45
  10. 2025Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworkscited by 37
  11. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  12. 2025R3: Robust Rubric-Agnostic Reward Modelscited by 22
38 more
  1. 2025Think-J: Learning to Think for Generative LLM-as-a-Judgecited by 20
  2. 2025Variation in Verification: Understanding Verification Dynamics in Large Language Modelscited by 18
  3. 2025ReviewRL: Towards Automated Scientific Review with RLcited by 17
  4. 2025Incentivizing Agentic Reasoning in LLM Judges via Tool-Integrated Reinforcement Learningcited by 16
  5. 2025Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflictcited by 15
  6. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  7. 2026AgentV-RL: Scaling Reward Modeling with Agentic Verifiercited by 12
  8. 2026Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Trainingcited by 11
  9. 2025J4R: Learning to Judge with Equivalent Initial State Group Relative Policy Optimizationcited by 9
  10. 2025Omni-Thinker: Scaling Multi-Task RL in LLMs with Hybrid Reward and Task Schedulingcited by 9
  11. 2025ReasonGRM: Enhancing Generative Reward Models through Large Reasoning Modelscited by 8
  12. 2025mR3: Multilingual Rubric-Agnostic Reward Reasoning Modelscited by 8
  13. 2026Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimizationcited by 8
  14. 2025CHARM: Calibrating Reward Models With Chatbot Arena Scorescited by 7
  15. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 7
  16. 2025Think Twice: Branch-and-Rethink Reasoning Reward Modelcited by 7
  17. 2026Generating and Refining Dynamic Evaluation Rubrics for LLM-as-a-Judgecited by 7
  18. 2025Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluatorscited by 6
  19. 2025Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimationcited by 6
  20. 2026Qworld: Question-Specific Evaluation Criteria for LLMscited by 6
  21. 2025Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimizationcited by 5
  22. 2025Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domainscited by 5
  23. 2025PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modelingcited by 5
  24. 2025Deep Ideation: Designing LLM Agents to Generate Novel Research Ideas on Scientific Concept Networkcited by 5
  25. 2026Grad2Reward: From Sparse Judgment to Dense Rewards for Improving Open-Ended LLM Reasoningcited by 5
  26. 2025Libra: Assessing and Improving Reward Model by Learning to Thinkcited by 4
  27. 2025VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressionscited by 4
  28. 2025CoSineVerifier: Tool-Augmented Answer Verification for Computation-Oriented Scientific Questionscited by 4
  29. 2026Aletheia: What Makes RLVR For Code Verifiers Tick?cited by 4
  30. 2025On the Shelf Life of Fine-Tuned LLM-Judges: Future-Proofing, Backward-Compatibility, and Question Generalizationcited by 3
  31. 2026CAMEL: Confidence-Gated Reflection for Reward Modelingcited by 3
  32. 2026Is Your LLM-as-a-Recommender Agent Trustable? LLMs' Recommendation is Easily Hacked by Biases (Preferences)cited by 3
  33. 2026RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domainscited by 3
  34. 2026Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modelingcited by 3
  35. 2025PulseReddit: A Novel Reddit Dataset for Benchmarking MAS in High-Frequency Cryptocurrency Tradingcited by 2
  36. 2025Agent-Testing Agent: A Meta-Agent for Automated Testing and Evaluation of Conversational AI Agentscited by 2
  37. 2025Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignmentcited by 2
  38. 2025OpenReward: Learning to Reward Long-form Agentic Tasks via Reinforcement Learningcited by 2

50 most cited of 76 All 76 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustness
  2. 2025Improve LLM-as-a-Judge Ability as a General Ability
  3. 2024From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks
  4. 2026SLMJury: Can Small Language Models Judge as Well as Large Ones?
  5. 2025Training Language Models to Reason Efficiently
  6. 2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
  7. 2026The Art of Efficient Reasoning: Data, Reward, and Optimization
  8. 2025ConciseRL: Conciseness-Guided Reinforcement Learning for Efficient Reasoning Models