mlsanityData to 14 Oct 2026

HexEval: An Evidence-Driven Hexagonal Framework for Multidimensional Scholar Assessment

arXiv 2608.10584 ↗First posted 11 Aug 2026cs.AI cs.DLcited by 0

Not on the map. Why?

Abstract (arXiv)

Scholar assessment plays a fundamental role in faculty recruitment, funding allocation, academic promotion, and talent discovery. Existing scholar assessment methods predominantly rely on bibliometric indicators and reputation proxies, while recent large language model (LLM)-based approaches mainly focus on evaluating individual research papers rather than comprehensively assessing scholars. We argue that scholar assessment should be formulated as an evidence-driven reasoning problem that jointly considers intrinsic research quality and externally verifiable scholarly behavior. To this end, we propose HexEval, an evidence-driven hexagonal framework for multidimensional scholar assessment. HexEval explicitly organizes scholar assessment into two complementary evidence layers. The intrinsic layer evaluates anonymized representative works along three dimensions, namely research rigor, methodological innovation, and scientific contribution, whereas the external layer characterizes scholars through knowledge translation, research coherence, and academic impact using heterogeneous evidence collected from GitHub, Lens, OpenAlex, and other publicly verifiable sources. Instead of producing opaque aggregate scores, HexEval preserves intermediate evidence, dimension-specific rationales, and verification signals throughout the evaluation process, enabling interpretable and auditable scholar profiles. Experiments across all six dimensions show dimension-dependent agreement with human or external reference criteria: structured calibration improves absolute agreement for intrinsic quality, while the external modules recover broad trajectory and ordinal impact signals. These results support evidence-driven reasoning over heterogeneous scholarly evidence as a promising paradigm for auditable AI-assisted scholar assessment, while exposing the coverage and attribution limitations of public scholarly data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 28 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

12 of 28 references.

References · 12

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  4. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  5. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  6. 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 177
  7. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 129
  8. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 100
  9. 2024LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewingcited by 62
  10. 2025Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025cited by 47
  11. 2024Evaluating Research Quality with Large Language Models: An Analysis of ChatGPT's Effectiveness with Different Settings and Inputscited by 7
  12. 2025Can ChatGPT evaluate research environments? Evidence from REF2021cited by 1

Built on by

Not cited yet.