mlsanityData to 7 Oct 2026

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

arXiv 2405.01535 ↗First posted 2 May 2024cs.CLcited by 431

Not on the map. Why?

Cited by 64 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Proprietary LMs such as GPT-4 are often employed to assess the quality of responses from various LMs. However, concerns including transparency, controllability, and affordability strongly motivate the development of open-source LMs specialized in evaluations. On the other hand, existing open evaluator LMs exhibit critical shortcomings: 1) they issue scores that significantly diverge from those assigned by humans, and 2) they lack the flexibility to perform both direct assessment and pairwise ranking, the two most prevalent forms of assessment. Additionally, they do not possess the ability to evaluate based on custom evaluation criteria, focusing instead on general attributes like helpfulness and harmlessness. To address these issues, we introduce Prometheus 2, a more powerful evaluator LM than its predecessor that closely mirrors human and GPT-4 judgements. Moreover, it is capable of processing both direct assessment and pair-wise ranking formats grouped with a user-defined evaluation criteria. On four direct assessment benchmarks and four pairwise ranking benchmarks, Prometheus 2 scores the highest correlation and agreement with humans and proprietary LM judges among all tested open evaluator LMs. Our models, code, and data are all publicly available at https://github.com/prometheus-eval/prometheus-eval.

Key facts. Cited by 431 papers in the map (data through 7 Oct 2026). Builds on 28 of its 45 references that are in the map. First posted on arXiv 2 May 2024.

Builds on

28 of 45 references.

References · 28

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  6. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  7. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  8. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  9. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  10. 2021Merging Models with Fisher-Weighted Averagingcited by 613
  11. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  12. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
16 more
  1. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  2. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  3. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  4. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  5. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  6. 2023FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Setscited by 198
  7. 2022Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Modelscited by 187
  8. 2024Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewardscited by 145
  9. 2024Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluationcited by 123
  10. 2020BLEU might be Guilty but References are not Innocentcited by 109
  11. 2024Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLMcited by 106
  12. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  13. 2023TIGERScore: Towards Building Explainable Metric for All Text Generation Taskscited by 69
  14. 2023Scaling Expert Language Models with Unsupervised Domain Discoverycited by 59
  15. 2022ColD Fusion: Collaborative Descent for Distributed Multitask Finetuningcited by 52
  16. 2022GEMv2: Multilingual NLG Benchmarking in a Single Line of Codecited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 94; 2025: 168; 2026: 169.

431 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  3. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 369
  4. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
  5. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
  6. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  7. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
  8. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 202
  9. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 155
  10. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 124
  11. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 119
  12. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
38 more
  1. 2024Can LLM be a Personalized Judge?cited by 99
  2. 2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 98
  3. 2024OffsetBias: Leveraging Debiased Data for Tuning Evaluatorscited by 94
  4. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  5. 2024Aligning to Thousands of Preferences via System Message Generalizationcited by 84
  6. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  7. 2025Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judgecited by 75
  8. 2025J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learningcited by 73
  9. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 72
  10. 2024Self-Generated Critiques Boost Reward Modeling for Language Modelscited by 69
  11. 2024Reinforcement Learning Enhanced LLMs: A Surveycited by 69
  12. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
  13. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 63
  14. 2024Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutorscited by 60
  15. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 58
  16. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  17. 2024Towards Lifelong Dialogue Agents via Timeline-based Memory Managementcited by 50
  18. 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutorscited by 47
  19. 2025ResearchQA: Evaluating Scholarly Question Answering at Scale Across 75 Fields with Survey-Mined Questions and Rubricscited by 44
  20. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  21. 2024Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversationscited by 43
  22. 2024Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Ratescited by 42
  23. 2025Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluatorscited by 42
  24. 2025Evaluating Scoring Bias in LLM-as-a-Judgecited by 42
  25. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  26. 2025RocketEval: Efficient Automated LLM Evaluation via Grading Checklistcited by 37
  27. 2025Improving LLM-as-a-Judge Inference with the Judgment Distributioncited by 36
  28. 2024CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generationcited by 35
  29. 2025Improve LLM-as-a-Judge Ability as a General Abilitycited by 35
  30. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  31. 2025Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judgecited by 35
  32. 2024Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructionscited by 34
  33. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 34
  34. 2024MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Modelscited by 34
  35. 2024Direct Judgement Preference Optimizationcited by 33
  36. 2024Finding Blind Spots in Evaluator LLMs with Interpretable Checklistscited by 32
  37. 2024LLM-as-a-Judge & Reward Model: What They Can and Cannot Docited by 32
  38. 2024CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judgescited by 32

50 most cited of 431 All 431 (JSON)