SkillVerse : Assessing and Enhancing LLMs with Tree Evaluation
Not on the map. Why?
Abstract (arXiv)
As language models evolve to tackle complex, multifaceted tasks, their evaluation must adapt to capture this intricacy. A granular, skill-specific understanding of model capabilities can empower researchers to make informed model development plans. In this paper, we introduce SkillVerse, an unsupervised tree-structured diagnosis framework for understanding model proficiency in specific abilities. With LLM as a judge, SkillVerse first critiques the model responses, and then organizes them into a hierarchical structure termed dendrogram. Given proficiency at arbitrary levels of granularity, SkillVerse is flexible to produce insights of behaviors of modern large models. We also demonstrate its efficacy in two downstream tasks: 1) improving model in-context learning by 25% using a tree-search algorithm to select more informative few-shot demonstrations, and 2) accurately predicting new model weaknesses with a 55% success rate, 22% higher than without SkillVerse.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 31 May 2025.
Builds on
20 of 29 references.
References · 20
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
- 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
- 2023Inverse Scaling: When Bigger Isn't Bettercited by 207
- 2023Shepherd: A Critic for Language Model Generationcited by 97
- 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 74
- 2024In-Context Principle Learning from Mistakescited by 44
- 2023LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedbackcited by 39
- 2024Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructionscited by 34
8 more
- 2024LLM-as-a-Judge & Reward Model: What They Can and Cannot Docited by 32
- 2021Semantics-Guided Contrastive Network for Zero-Shot Object detectioncited by 26
- 2024C-ICL: Contrastive In-context Learning for Information Extractioncited by 23
- 2024Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisonscited by 22
- 2023QualEval: Qualitative Evaluation for Model Improvementcited by 11
- 2024Unearthing Skill-Level Insights for Understanding Trade-Offs of Foundation Modelscited by 9
- 2024Rationale-Aware Answer Verification by Pairwise Self-Evaluationcited by 8
- 2024In-context Contrastive Learning for Event Causality Identificationcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.