Objective Metrics for Evaluating Large Language Models Using External Data Sources
Not on the map. Why?
Abstract (arXiv)
Evaluating the performance of Large Language Models (LLMs) is a critical yet challenging task, particularly when aiming to avoid subjective assessments. This paper proposes a framework for leveraging subjective metrics derived from the class textual materials across different semesters to assess LLM outputs across various tasks. By utilizing well-defined benchmarks, factual datasets, and structured evaluation pipelines, the approach ensures consistent, reproducible, and bias-minimized measurements. The framework emphasizes automation and transparency in scoring, reducing reliance on human interpretation while ensuring alignment with real-world applications. This method addresses the limitations of subjective evaluation methods, providing a scalable solution for performance assessment in educational, scientific, and other high-stakes domains.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 29 references that are in the map. First posted on arXiv 1 Aug 2025.
Builds on
10 of 29 references.
References · 10
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023A Survey on Evaluation of Large Language Modelscited by 1,242
- 2023PaLM 2 Technical Reportcited by 1,162
- 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 418
- 2023A Survey on Large Language Models for Recommendationcited by 379
- 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155
- 2024Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluationcited by 58
- 2023Language Models are Few-shot Learners for Prognostic Predictioncited by 37
- 2024Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Studycited by 26
- 2020Personalized Multimodal Feedback Generation in Educationcited by 4
Built on by
Not cited yet.