Oracle-Checker Scheme for Evaluating a Generative Large Language Model
Not on the map. Why?
Abstract (arXiv)
This work presents a novel approach called oracle-checker scheme for evaluating the answer given by a generative large language model (LLM). Two types of checkers are presented. The first type of checker follows the idea of property testing. The second type of checker follows the idea of program checking. Their applications are demonstrated in two separate contexts, entity extraction and paraphrase decision, respectively.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 24 references that are in the map. First posted on arXiv 6 May 2024.
Builds on
5 of 24 references.
References · 5
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
- 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 481
- 2019DocRED: A Large-Scale Document-Level Relation Extraction Datasetcited by 239
- 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 131
Built on by
Not cited yet.