mlsanityData to 7 Oct 2026

A Proposed S.C.O.R.E. Evaluation Framework for Large Language Models : Safety, Consensus, Objectivity, Reproducibility and Explainability

arXiv 2407.07666 ↗First posted 10 Jul 2024cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

A comprehensive qualitative evaluation framework for large language models (LLM) in healthcare that expands beyond traditional accuracy and quantitative metrics needed. We propose 5 key aspects for evaluation of LLMs: Safety, Consensus, Objectivity, Reproducibility and Explainability (S.C.O.R.E.). We suggest that S.C.O.R.E. may form the basis for an evaluation framework for future LLM-based models that are safe, reliable, trustworthy, and ethical for healthcare and clinical applications.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 14 of its 23 references that are in the map. First posted on arXiv 10 Jul 2024.

Builds on

14 of 23 references.

References · 14

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  7. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  8. 2020Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlationcited by 420
  9. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  10. 2020USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generationcited by 186
  11. 2016Exploring the Limits of Language Modelingcited by 172
  12. 2023LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Modelscited by 100
2 more
  1. 2024A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industrycited by 17
  2. 2024Fine-tuning Large Language Model (LLM) Artificial Intelligence Chatbots in Ophthalmology and LLM-based evaluation using GPT-4cited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applicationscited by 25
  2. 2025MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systemscited by 8
  3. 2025Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applicationscited by 0