# Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses

arXiv: https://arxiv.org/abs/2509.16093 · HTML: https://mlsanity.com/paper/2509.16093/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Large Language Models in Law: A Survey](https://mlsanity.com/paper/2312.03718/) (2023)
- [Evaluation Ethics of LLMs in Legal Domain](https://mlsanity.com/paper/2403.11152/) (2024)

## Built on by (most cited first, 8 of 8)

- [All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection](https://mlsanity.com/paper/2601.04160/) (2026)
- [From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape](https://mlsanity.com/paper/2606.08625/) (2026)
- [C2: Scalable Rubric-Augmented Reward Modeling from Binary Preferences](https://mlsanity.com/paper/2604.13618/) (2026)
- [AI-assisted Protocol Information Extraction For Improved Accuracy and Efficiency in Clinical Trial Workflows](https://mlsanity.com/paper/2602.00052/) (2026)
- [Agentic systems for breast cancer treatment recommendations](https://mlsanity.com/paper/2607.12051/) (2026)
- [SESSE: Sketch, Expand, Sort, Summarize, Evaluate -- LLM-as-Judge Evaluation via Structured Decomposition](https://mlsanity.com/paper/2608.18303/) (2026)
- [VICT: Verifier-Instrumented Credit Tracing for Long-Horizon LLM Agent Reinforcement Learning](https://mlsanity.com/paper/2608.28128/) (2026)
- [CoCA: Criterion-Aware Credit Assignment for Rubric-Based Reinforcement Learning](https://mlsanity.com/paper/2610.17733/) (2026)

## Abstract (arXiv)

Evaluating long-form answers in high-stakes domains such as law or medicine remains a fundamental challenge. Standard metrics like BLEU and ROUGE fail to capture semantic correctness, and current LLM-based evaluators often reduce nuanced aspects of answer quality into a single undifferentiated score. We introduce DeCE, a decomposed LLM evaluation framework that separates precision (factual accuracy and relevance) and recall (coverage of required concepts), using instance-specific criteria automatically extracted from gold answer requirements. DeCE is model-agnostic and domain-general, requiring no predefined taxonomies or handcrafted rubrics. We instantiate DeCE to evaluate different LLMs on a real-world legal QA task involving multi-jurisdictional reasoning and citation grounding. DeCE achieves substantially stronger correlation with expert judgments (r=0.78), compared to traditional metrics (r=0.12), pointwise LLM scoring (r=0.35), and modern multidimensional evaluators (r=0.48). It also reveals interpretable trade-offs: generalist models favor recall, while specialized models favor precision. Importantly, only 11.95% of LLM-generated criteria required expert revision, underscoring DeCE's scalability. DeCE offers an interpretable and actionable LLM evaluation framework in expert domains.