# LLMs and the Madness of Crowds

arXiv: https://arxiv.org/abs/2411.01539 · HTML: https://mlsanity.com/paper/2411.01539/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 9 references that are in the map.
- First posted on arXiv 3 Nov 2024.

## Builds on (most cited first, 2 of 2)

- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [A Survey on Mixture of Experts in Large Language Models](https://mlsanity.com/paper/2407.06204/) (2024)

## Built on by (most cited first, 2 of 2)

- [ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Models](https://mlsanity.com/paper/2601.15812/) (2026)
- [Team Anotheroption at SemEval-2025 Task 8: Bridging the Gap Between Open-Source and Proprietary LLMs in Table QA](https://mlsanity.com/paper/2506.09657/) (2025)

## Abstract (arXiv)

We investigate the patterns of incorrect answers produced by large language models (LLMs) during evaluation. These errors exhibit highly non-intuitive behaviors unique to each model. By analyzing these patterns, we measure the similarities between LLMs and construct a taxonomy that categorizes them based on their error correlations. Our findings reveal that the incorrect responses are not randomly distributed but systematically correlated across models, providing new insights into the underlying structures and relationships among LLMs.