# Do Large Language Models have Problem-Solving Capability under Incomplete Information Scenarios?

arXiv: https://arxiv.org/abs/2409.14762 · HTML: https://mlsanity.com/paper/2409.14762/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2024.

## Built on by (most cited first, 7 of 7)

- [The Homogenizing Effect of Large Language Models on Human Expression and Thought](https://mlsanity.com/paper/2508.01491/) (2025)
- [PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilities](https://mlsanity.com/paper/2504.14773/) (2025)
- [Enhancing Exchange Rate Forecasting with Explainable Deep Learning Models](https://mlsanity.com/paper/2410.19241/) (2024)
- [Probing the Lack of Stable Internal Beliefs in LLMs](https://mlsanity.com/paper/2603.25187/) (2026)
- [GuessingGame: Measuring the Informativeness of Open-Ended Questions in Large Language Models](https://mlsanity.com/paper/2509.19593/) (2025)
- [Pre-trained Graphformer-based Ranking at Web-scale Search (Extended Abstract)](https://mlsanity.com/paper/2409.16590/) (2024)
- [Generative Pre-trained Ranking Model with Over-parameterization at Web-Scale (Extended Abstract)](https://mlsanity.com/paper/2409.16594/) (2024)

## Abstract (arXiv)

The evaluation of the problem-solving capability under incomplete information scenarios of Large Language Models (LLMs) is increasingly important, encompassing capabilities such as questioning, knowledge search, error detection, and path planning. Current research mainly focus on LLMs' problem-solving capability such as ``Twenty Questions''. However, these kinds of games do not require recognizing misleading cues which are necessary in the incomplete information scenario. Moreover, the existing game such as ``Who is undercover'' are highly subjective, making it challenging for evaluation. Therefore, in this paper, we introduce a novel game named BrainKing based on the ``Who is undercover'' and ``Twenty Questions'' for evaluating LLM capabilities under incomplete information scenarios. It requires LLMs to identify target entities with limited yes-or-no questions and potential misleading answers. By setting up easy, medium, and hard difficulty modes, we comprehensively assess the performance of LLMs across various aspects. Our results reveal the capabilities and limitations of LLMs in BrainKing, providing significant insights of LLM problem-solving levels.