# Zero-Shot Belief: A Hard Problem for LLMs

arXiv: https://arxiv.org/abs/2502.08777 · HTML: https://mlsanity.com/paper/2502.08777/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 30 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 8 of 8)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Neural models of factuality](https://mlsanity.com/paper/1804.02472/) (2018)
- [Lexicosyntactic Inference in Neural Models](https://mlsanity.com/paper/1808.06232/) (2018)
- [He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics](https://mlsanity.com/paper/2107.00807/) (2021)
- [Graph based Neural Networks for Event Factuality Prediction using Syntactic and Semantic Structures](https://mlsanity.com/paper/1907.03227/) (2019)
- [MAVEN-Fact: A Large-scale Event Factuality Detection Dataset](https://mlsanity.com/paper/2407.15352/) (2024)

## Abstract (arXiv)

We present two LLM-based approaches to zero-shot source-and-target belief prediction on FactBank: a unified system that identifies events, sources, and belief labels in a single pass, and a hybrid approach that uses a fine-tuned DeBERTa tagger for event detection. We show that multiple open-sourced, closed-source, and reasoning-based LLMs struggle with the task. Using the hybrid approach, we achieve new state-of-the-art results on FactBank and offer a detailed error analysis. Our approach is then tested on the Italian belief corpus ModaFact.