# Evolution and compression in LLMs: On the emergence of human-aligned categorization

arXiv: https://arxiv.org/abs/2509.08093 · HTML: https://mlsanity.com/paper/2509.08093/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 50 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Builds on (most cited first, 8 of 8)

- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [Compositional Languages Emerge in a Neural Iterated Learning Model](https://mlsanity.com/paper/2002.01365/) (2020)
- [Large language models predict human sensory judgments across six modalities](https://mlsanity.com/paper/2302.01308/) (2023)
- [Eliciting the Priors of Large Language Models using Iterated In-Context Learning](https://mlsanity.com/paper/2406.01860/) (2024)
- [Bias Amplification in Language Model Evolution: An Iterated Learning Perspective](https://mlsanity.com/paper/2404.04286/) (2024)
- [Semantic categories of artifacts and animals reflect efficient coding](https://mlsanity.com/paper/1905.04562/) (2019)
- [Cultural evolution via iterated learning and communication explains efficient color naming systems](https://mlsanity.com/paper/2305.10154/) (2023)
- [Searching for Structure: Investigating Emergent Communication with Large Language Models](https://mlsanity.com/paper/2412.07646/) (2024)

## Built on by (most cited first, 1 of 1)

- [Where Should Language Sit in a Multimodal Model? Lessons from What Language Does to Human Perception and Cognition](https://mlsanity.com/paper/2609.07474/) (2026)

## Abstract (arXiv)

Converging evidence suggests that human systems of semantic categories achieve near-optimal compression via the Information Bottleneck (IB) complexity-accuracy tradeoff. Large language models (LLMs) are not trained for this objective, which raises the question: are LLMs capable of evolving efficient human-aligned semantic systems? To address this question, we focus on color categorization -- a key testbed of cognitive theories of categorization with uniquely rich human data -- and replicate with LLMs two influential human studies. First, we conduct an English color-naming study, showing that LLMs vary widely in their complexity and English-alignment, with larger instruction-tuned models achieving better alignment and IB-efficiency. Second, to test whether these LLMs simply mimic patterns in their training data or actually exhibit a human-like inductive bias toward IB-efficiency, we simulate cultural evolution of pseudo color-naming systems in LLMs via a method we refer to as Iterated in-Context Language Learning (IICLL). We find that akin to humans, LLMs iteratively restructure initially random systems towards greater IB-efficiency. However, only a model with strongest in-context capabilities (Gemini 2.0) is able to recapitulate the wide range of near-optimal IB-tradeoffs observed in humans, while other state-of-the-art models converge to low-complexity solutions. These findings demonstrate how human-aligned semantic categories can emerge in LLMs via the same fundamental principle that underlies semantic efficiency in humans.