# From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning

arXiv: https://arxiv.org/abs/2505.17117 · HTML: https://mlsanity.com/paper/2505.17117/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 24 of its 52 references that are in the map.
- First posted on arXiv 21 May 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [Large Concept Models: Language Modeling in a Sentence Representation Space](https://mlsanity.com/paper/2412.08821/) (2024)
- [The Geometry of Concepts: Sparse Autoencoder Feature Structure](https://mlsanity.com/paper/2410.19750/) (2024)
- [Do language models learn typicality judgments from text?](https://mlsanity.com/paper/2105.02987/) (2021)
- [Semantic categories of artifacts and animals reflect efficient coding](https://mlsanity.com/paper/1905.04562/) (2019)
- [Towards Concept-Aware Large Language Models](https://mlsanity.com/paper/2311.01866/) (2023)
- [LLM-assisted Concept Discovery: Automatically Identifying and Explaining Neuron Functions](https://mlsanity.com/paper/2406.08572/) (2024)
- [Building, Reusing, and Generalizing Abstract Representations from Concrete Sequences](https://mlsanity.com/paper/2410.21332/) (2024)

## Built on by (most cited first, 25 of 25)

- [An Information Theoretic Perspective on Agentic System Design](https://mlsanity.com/paper/2512.21720/) (2025)
- [Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impact](https://mlsanity.com/paper/2507.00951/) (2025)
- [The Other Mind: How Language Models Exhibit Human Temporal Cognition](https://mlsanity.com/paper/2507.15851/) (2025)
- [What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests](https://mlsanity.com/paper/2507.11128/) (2025)
- [Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs](https://mlsanity.com/paper/2511.01202/) (2025)
- [Beyond Tokens: Concept-Level Training Objectives for LLMs](https://mlsanity.com/paper/2601.11791/) (2026)
- [Exploring Human-AI Conceptual Alignment through the Prism of Chess](https://mlsanity.com/paper/2510.26025/) (2025)
- [Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification](https://mlsanity.com/paper/2601.20742/) (2026)
- [Training deep physical neural networks with local physical information bottleneck](https://mlsanity.com/paper/2602.09569/) (2026)
- [Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models](https://mlsanity.com/paper/2603.29123/) (2026)
- [Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMs](https://mlsanity.com/paper/2606.27378/) (2026)
- [What Do You Mean? Exploring How Humans and AI Interact with Symbols and Meanings in Their Interactions](https://mlsanity.com/paper/2510.05378/) (2025)
- [Mechanistic Origin of Moral Indifference in Language Models](https://mlsanity.com/paper/2603.15615/) (2026)
- [PROMPT2BOX:Improving LLM Weakness Discovery and Specificity Estimation by Uncovering Entailment Structure among Prompts](https://mlsanity.com/paper/2603.21438/) (2026)
- [Gyan: An Explainable Neuro-Symbolic Language Model](https://mlsanity.com/paper/2605.04759/) (2026)
- [Beyond a Bag of Features: Set-Level Instability in Sparse Autoencoders](https://mlsanity.com/paper/2608.11197/) (2026)
- [Position: We Need Practical AI Alignment Methods to Mirror Human Reasoning](https://mlsanity.com/paper/2608.12372/) (2026)
- [Is this Citation on Point?](https://mlsanity.com/paper/2608.12571/) (2026)
- [BayesPrompt: human readable prompts that make sense](https://mlsanity.com/paper/2608.17866/) (2026)
- [AI Models Can Predict and Collaboratively Modulate Human Memory Search](https://mlsanity.com/paper/2608.26152/) (2026)
- [Representational alignment yields generalizable safety in language models](https://mlsanity.com/paper/2609.04022/) (2026)
- [Why We Care About Understanding: Competence through Predictive Compression](https://mlsanity.com/paper/2609.04962/) (2026)
- [Where Should Language Sit in a Multimodal Model? Lessons from What Language Does to Human Perception and Cognition](https://mlsanity.com/paper/2609.07474/) (2026)
- [Toward a Unified Mathematics of Concepts](https://mlsanity.com/paper/2609.24554/) (2026)
- [When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors](https://mlsanity.com/paper/2610.13910/) (2026)

## Abstract (arXiv)

Humans organize knowledge into compact conceptual categories that balance compression with semantic richness. Large Language Models (LLMs) exhibit impressive linguistic abilities, but whether they navigate this same compression-meaning trade-off remains unclear. We apply an Information Bottleneck framework to compare human conceptual structure with embeddings from 40+ LLMs using classic categorization benchmarks. We find that LLMs broadly align with human category boundaries, yet fall short on fine-grained semantic distinctions. Unlike humans, who maintain ``inefficient'' representations that preserve contextual nuance, LLMs aggressively compress, achieving more optimal information-theoretic compression at the cost of semantic richness. Surprisingly, encoder models outperform much larger decoder models in human alignment, suggesting that understanding and generation rely on distinct representational mechanisms. Training-dynamics analysis reveals a two-phase trajectory: rapid initial concept formation followed by architectural reorganization, during which semantic processing migrates from deep to mid-network layers as the model discovers increasingly efficient, sparser encodings. These divergent strategies, where LLMs optimize for compression and humans for adaptive utility, reveal fundamental differences between artificial and natural intelligence. This highlights the need for models that preserve the conceptual ``inefficiencies'' essential for human-like understanding.