# Granting GPT-4 License and Opportunity: Enhancing Accuracy and Confidence Estimation for Few-Shot Event Detection

arXiv: https://arxiv.org/abs/2408.00914 · HTML: https://mlsanity.com/paper/2408.00914/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 8 of 8)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Information Extraction in Low-Resource Scenarios: Survey and Perspective](https://mlsanity.com/paper/2202.08063/) (2022)
- [The Confidence-Competence Gap in Large Language Models: A Cognitive Study](https://mlsanity.com/paper/2309.16145/) (2023)
- [Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty](https://mlsanity.com/paper/2309.03433/) (2023)
- [Massively Multi-Lingual Event Understanding: Extraction, Visualization, and Search](https://mlsanity.com/paper/2305.10561/) (2023)

## Abstract (arXiv)

Large Language Models (LLMs) such as GPT-4 have shown enough promise in the few-shot learning context to suggest use in the generation of "silver" data and refinement of new ontologies through iterative application and review. Such workflows become more effective with reliable confidence estimation. Unfortunately, confidence estimation is a documented weakness of models such as GPT-4, and established methods to compensate require significant additional complexity and computation. The present effort explores methods for effective confidence estimation with GPT-4 with few-shot learning for event detection in the BETTER ontology as a vehicle. The key innovation is expanding the prompt and task presented to GPT-4 to provide License to speculate when unsure and Opportunity to quantify and explain its uncertainty (L&O). This approach improves accuracy and provides usable confidence measures (0.759 AUC) with no additional machinery.