# PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning

arXiv: https://arxiv.org/abs/2511.11562 · HTML: https://mlsanity.com/paper/2511.11562/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 14 Nov 2025.

## Builds on (most cited first, 20 of 20)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://mlsanity.com/paper/2505.08775/) (2025)
- [Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains](https://mlsanity.com/paper/2507.17746/) (2025)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://mlsanity.com/paper/2210.03849/) (2022)
- [ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems](https://mlsanity.com/paper/2505.11831/) (2025)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [Reinforcement Learning with Rubric Anchors](https://mlsanity.com/paper/2508.12790/) (2025)
- [Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks](https://mlsanity.com/paper/2410.20266/) (2024)
- [SWE-Lancer: Can Frontier LLMs Earn 1 Million from Real-World Freelance Software Engineering?](https://mlsanity.com/paper/2502.12115/) (2025)
- [Determinants of LLM-assisted Decision-Making](https://mlsanity.com/paper/2402.17385/) (2024)
- [Online Rubrics Elicitation from Pairwise Comparisons](https://mlsanity.com/paper/2510.07284/) (2025)
- [FinanceMath: Knowledge-Intensive Math Reasoning in Finance Domains](https://mlsanity.com/paper/2311.09797/) (2023)
- [FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging](https://mlsanity.com/paper/2506.05828/) (2025)
- [Remote Labor Index: Measuring AI Automation of Remote Work](https://mlsanity.com/paper/2510.26787/) (2025)
- [GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citations](https://mlsanity.com/paper/2505.17267/) (2025)
- [The AI Productivity Index (APEX)](https://mlsanity.com/paper/2509.25721/) (2025)

## Built on by (most cited first, 30 of 32)

- [Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Training](https://mlsanity.com/paper/2602.01511/) (2026)
- [Agentic Rubrics as Contextual Verifiers for SWE Agents](https://mlsanity.com/paper/2601.04171/) (2026)
- [Autodata: An agentic data scientist to create high quality synthetic data](https://mlsanity.com/paper/2606.25996/) (2026)
- [PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice](https://mlsanity.com/paper/2601.16669/) (2026)
- [M^star: Every Task Deserves Its Own Memory Harness](https://mlsanity.com/paper/2604.11811/) (2026)
- [JobBench: Aligning Agent Work With Human Will](https://mlsanity.com/paper/2605.26329/) (2026)
- [ShoppingComp: Are LLMs Really Ready for Your Shopping Cart?](https://mlsanity.com/paper/2511.22978/) (2025)
- [BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents](https://mlsanity.com/paper/2606.03829/) (2026)
- [SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution](https://mlsanity.com/paper/2605.08366/) (2026)
- [Visual Preference Optimization with Rubric Rewards](https://mlsanity.com/paper/2604.13029/) (2026)
- [TraderBench: How Robust Are AI Agents in Adversarial Capital Markets?](https://mlsanity.com/paper/2603.00285/) (2026)
- [ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning](https://mlsanity.com/paper/2605.23454/) (2026)
- [Reinforcement Learning with Robust Rubric Rewards](https://mlsanity.com/paper/2605.30244/) (2026)
- [BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows](https://mlsanity.com/paper/2604.11304/) (2026)
- [Shopping Reasoning Bench: An Expert-Authored Benchmark for Multi-Turn Conversational Shopping Assistants](https://mlsanity.com/paper/2606.12608/) (2026)
- [Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation](https://mlsanity.com/paper/2608.24138/) (2026)
- [Thomson: Continual Learning of Frontier Models for SovereignAI](https://mlsanity.com/paper/2608.27147/) (2026)
- [Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2601.15267/) (2026)
- [PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology](https://mlsanity.com/paper/2603.01343/) (2026)
- [SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks](https://mlsanity.com/paper/2603.10002/) (2026)
- [On Predicting the Post-training Potential of Pre-trained LLMs](https://mlsanity.com/paper/2605.11978/) (2026)
- [Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck](https://mlsanity.com/paper/2608.18931/) (2026)
- [Legal Research Bench: Measuring End-to-End Reliability in Long-Horizon Legal Research Agents](https://mlsanity.com/paper/2610.00609/) (2026)
- [NuclearQAv2: A Structured Benchmark for Evaluating Domain-Science Competence in Large Language Models](https://mlsanity.com/paper/2606.27047/) (2026)
- [CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data](https://mlsanity.com/paper/2607.16122/) (2026)
- [Cross-Domain Hybrid OPD for Generalizable Search Agents](https://mlsanity.com/paper/2608.02101/) (2026)
- [ExecRubrics: Executable Tool-Augmented Rubrics for Verifiable and Efficient Long-Form Evaluation](https://mlsanity.com/paper/2608.22559/) (2026)
- [WinSyn: An Automated Pipeline for Realistic Enterprise Question-Answering Evaluation](https://mlsanity.com/paper/2609.12171/) (2026)
- [A Cheap Verifier is Good Enough: LLM Post-training is Robust to Erroneous Rewards](https://mlsanity.com/paper/2609.33467/) (2026)
- [FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents](https://mlsanity.com/paper/2609.35744/) (2026)

## Abstract (arXiv)

Frontier model progress is often measured by academic benchmarks, which offer a limited view of performance in real-world professional contexts. Existing evaluations often fail to assess open-ended, economically consequential tasks in high-stakes domains like Legal and Finance, where practical returns are paramount. To address this, we introduce Professional Reasoning Bench (PRBench), a realistic, open-ended, and difficult benchmark of real-world problems in Finance and Law. We open-source its 1,100 expert-authored tasks and 19,356 expert-curated criteria, making it, to our knowledge, the largest public, rubric-based benchmark for both legal and finance domains. We recruit 182 qualified professionals, holding JDs, CFAs, or 6+ years of experience, who contributed tasks inspired by their actual workflows. This process yields significant diversity, with tasks spanning 114 countries and 47 US jurisdictions. Our expert-curated rubrics are validated through a rigorous quality pipeline, including independent expert validation. Subsequent evaluation of 20 leading models reveals substantial room for improvement, with top scores of only 0.39 (Finance) and 0.37 (Legal) on our Hard subsets. We further catalog associated economic impacts of the prompts and analyze performance using human-annotated rubric categories. Our analysis shows that models with similar overall scores can diverge significantly on specific capabilities. Common failure modes include inaccurate judgments, a lack of process transparency and incomplete reasoning, highlighting critical gaps in their reliability for professional adoption.