# Probing LLM World Models: Enhancing Guesstimation with Wisdom of Crowds Decoding

arXiv: https://arxiv.org/abs/2501.17310 · HTML: https://mlsanity.com/paper/2501.17310/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)

## Builds on (most cited first, 19 of 19)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models](https://mlsanity.com/paper/2406.01584/) (2024)
- [Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models](https://mlsanity.com/paper/2404.03622/) (2024)
- [Fairness in Large Language Models: A Taxonomic Survey](https://mlsanity.com/paper/2404.01349/) (2024)
- [Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs](https://mlsanity.com/paper/2404.07449/) (2024)
- [When is a crowd wise?](https://mlsanity.com/paper/1406.7563/) (2014)
- [When is the consistent prediction likely to be a correct prediction?](https://mlsanity.com/paper/2407.05778/) (2024)
- [Enhancing the Spatial Awareness Capability of Multi-Modal Large Language Model](https://mlsanity.com/paper/2310.20357/) (2023)

## Built on by (most cited first, 4 of 4)

- [LAMP: Extracting Local Decision Surfaces From Large Language Models](https://mlsanity.com/paper/2505.11772/) (2025)
- [Future Querying: Can LLMs Serve as Implicit Medical World Models?](https://mlsanity.com/paper/2608.23248/) (2026)
- [Diverse by Reasoning: Harnessing the Wisdom of LLM Crowds for Future Prediction](https://mlsanity.com/paper/2608.24001/) (2026)
- [Multi-agent Scaling Across Disjunctive and Compensatory Tasks](https://mlsanity.com/paper/2609.31563/) (2026)

## Abstract (arXiv)

Guesstimation -- the task of making approximate quantitative estimates about objects or events -- is a common real-world skill, yet remains underexplored in large language model (LLM) research. We introduce three guesstimation datasets: MARBLES, FUTURE, and ELECPRED, spanning physical estimation (e.g., how many marbles fit in a cup) to abstract predictions (e.g., the 2024 U.S. presidential election). Inspired by the social science concept of Wisdom of Crowds (WOC)- where the median of multiple estimates improves accuracy-we propose WOC decoding for LLMs. We replicate WOC effects in human participants and find that LLMs exhibit similar benefits: median aggregation across sampled responses consistently improves accuracy over greedy decoding, self-consistency decoding, and mean decoding. This suggests that LLMs encode a world model that supports approximate reasoning. Our results position guesstimation as a useful probe of LLM world knowledge and highlight WOC decoding as a strategy for enhancing LLM guesstimation performance on real-world tasks.