mlsanityData to 14 Oct 2026

Evalverse: Unified and Accessible Library for Large Language Model Evaluation

arXiv 2404.00943 ↗First posted 1 Apr 2024cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper introduces Evalverse, a novel library that streamlines the evaluation of Large Language Models (LLMs) by unifying disparate evaluation tools into a single, user-friendly framework. Evalverse enables individuals with limited knowledge of artificial intelligence to easily request LLM evaluations and receive detailed reports, facilitated by an integration with communication platforms like Slack. Thus, Evalverse serves as a powerful tool for the comprehensive assessment of LLMs, offering both researchers and practitioners a centralized and easily accessible evaluation framework. Finally, we also provide a demo video for Evalverse, showcasing its capabilities and implementation in a two-minute format.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 1 Apr 2024.

Builds on

24 of 34 references.

References · 24

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  11. 2023Qwen Technical Reportcited by 3,344
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
12 more
  1. 2023A Survey of Large Language Modelscited by 2,576
  2. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  5. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 274
  6. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  7. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  8. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  9. 2024FOFO: A Benchmark to Evaluate LLMs' Format-Following Capabilitycited by 80
  10. 2023FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasetscited by 69
  11. 2022LegalBench: Prototyping a Collaborative Benchmark for Legal Reasoningcited by 17
  12. 2024Evaluation of General Large Language Models in Contextually Assessing Semantic Concepts Extracted from Adult Critical Care Electronic Health Record Notescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2024Do LLMs "know" internally when they follow instructions?cited by 26
  2. 2024Do LLMs estimate uncertainty well in instruction-following?cited by 26
  3. 2024Benchmarking LLMs for Environmental Review and Permittingcited by 4
  4. 2025Redefining Evaluation Standards: A Unified Framework for Evaluating the Korean Capabilities of Language Modelscited by 1
  5. 2026A Composable Evaluation System for Reproducible Omni-Modal Foundation Model Evaluationcited by 0