# BSBench: will your LLM find the largest prime number?

arXiv: https://arxiv.org/abs/2506.04535 · HTML: https://mlsanity.com/paper/2506.04535/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2025.

## Abstract (arXiv)

We propose that benchmarking LLMs on questions which have no reasonable answer actually isn't as silly as it sounds. We also present a benchmark that allows such testing and a method to modify the existing datasets, and discover that existing models demonstrate a performance far from the perfect on such questions. Our code and data artifacts are available at https://github.com/L3G5/impossible-bench