# The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches

arXiv: https://arxiv.org/abs/2406.03339 · HTML: https://mlsanity.com/paper/2406.03339/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Builds on (most cited first, 16 of 16)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2311.09476/) (2023)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [Approximating Online Human Evaluation of Social Chatbots with Prompting](https://mlsanity.com/paper/2304.05253/) (2023)
- [Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systems](https://mlsanity.com/paper/2212.09180/) (2022)
- [Evaluating Chatbots to Promote Users' Trust -- Practices and Open Problems](https://mlsanity.com/paper/2309.05680/) (2023)

## Built on by (most cited first, 18 of 18)

- [Language agents achieve superhuman synthesis of scientific knowledge](https://mlsanity.com/paper/2409.13740/) (2024)
- [Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts](https://mlsanity.com/paper/2503.09347/) (2025)
- [A Technical Survey of Reinforcement Learning Techniques for Large Language Models](https://mlsanity.com/paper/2507.04136/) (2025)
- [Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue](https://mlsanity.com/paper/2409.08330/) (2024)
- [A review of faithfulness metrics for hallucination assessment in Large Language Models](https://mlsanity.com/paper/2501.00269/) (2024)
- [Is artificial intelligence still intelligence? LLMs generalize to novel adjective-noun pairs, but don't mimic the full human distribution](https://mlsanity.com/paper/2410.17482/) (2024)
- [Generating a Low-code Complete Workflow via Task Decomposition and RAG](https://mlsanity.com/paper/2412.00239/) (2024)
- [Retrospective Comparative Analysis of Prostate Cancer In-Basket Messages: Responses from Closed-Domain LLM vs. Clinical Teams](https://mlsanity.com/paper/2409.18290/) (2024)
- [From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap](https://mlsanity.com/paper/2410.20791/) (2024)
- [Integrating Personality into Digital Humans: A Review of LLM-Driven Approaches for Virtual Reality](https://mlsanity.com/paper/2503.16457/) (2025)
- [COMPASS: A Framework for Evaluating Organization-Specific Policy Alignment in LLMs](https://mlsanity.com/paper/2601.01836/) (2026)
- [CASE: An Agentic AI Framework for Enhancing Scam Intelligence in Digital Payments](https://mlsanity.com/paper/2508.19932/) (2025)
- [Poison Once, Refuse Forever: Weaponizing Alignment for Injecting Bias in LLMs](https://mlsanity.com/paper/2508.20333/) (2025)
- [Plausibility as Failure: How LLMs and Humans Co-Construct Epistemic Error](https://mlsanity.com/paper/2512.16750/) (2025)
- [A Cross-Lingual Statutory Article Retrieval Dataset for Taiwan Legal Studies](https://mlsanity.com/paper/2410.11450/) (2024)
- [Value-Spectrum: Quantifying Preferences of Vision-Language Models via Value Decomposition in Social Media Contexts](https://mlsanity.com/paper/2411.11479/) (2024)
- [Metric assessment protocol in the context of answer fluctuation on MCQ tasks](https://mlsanity.com/paper/2507.15581/) (2025)
- [Advancing Academic Chatbots: Evaluation of Non Traditional Outputs](https://mlsanity.com/paper/2512.00991/) (2025)

## Abstract (arXiv)

Chatbots have been an interesting application of natural language generation since its inception. With novel transformer based Generative AI methods, building chatbots have become trivial. Chatbots which are targeted at specific domains for example medicine and psychology are implemented rapidly. This however, should not distract from the need to evaluate the chatbot responses. Especially because the natural language generation community does not entirely agree upon how to effectively evaluate such applications. With this work we discuss the issue further with the increasingly popular LLM based evaluations and how they correlate with human evaluations. Additionally, we introduce a comprehensive factored evaluation mechanism that can be utilized in conjunction with both human and LLM-based evaluations. We present the results of an experimental evaluation conducted using this scheme in one of our chatbot implementations which consumed educational reports, and subsequently compare automated, traditional human evaluation, factored human evaluation, and factored LLM evaluation. Results show that factor based evaluation produces better insights on which aspects need to be improved in LLM applications and further strengthens the argument to use human evaluation in critical spaces where main functionality is not direct retrieval.