# Cognitive Load Traces as Symbolic and Visual Accounts of Deep Model Cognition

arXiv: https://arxiv.org/abs/2510.15980 · HTML: https://mlsanity.com/paper/2510.15980/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation](https://mlsanity.com/paper/2505.20353/) (2025)
- [Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework](https://mlsanity.com/paper/2409.11585/) (2024)
- [MT2ST: Adaptive Multi-Task to Single-Task Learning](https://mlsanity.com/paper/2406.18038/) (2024)
- [LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inference](https://mlsanity.com/paper/2406.19657/) (2024)
- [HADES: Hardware Accelerated Decoding for Efficient Speculation in Large Language Models](https://mlsanity.com/paper/2412.19925/) (2024)
- [Breaking Silos: Adaptive Model Fusion Unlocks Better Time Series Forecasting](https://mlsanity.com/paper/2505.18442/) (2025)
- [Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling](https://mlsanity.com/paper/2504.15296/) (2025)
- [Cloud-Based AI Systems: Leveraging Large Language Models for Intelligent Fault Detection and Autonomous Self-Healing](https://mlsanity.com/paper/2505.11743/) (2025)
- [Cross-Cloud Data Privacy Protection: Optimizing Collaborative Mechanisms of AI Systems by Integrating Federated Learning and LLMs](https://mlsanity.com/paper/2505.13292/) (2025)
- [TinyServe: Query-Aware Cache Selection for Efficient LLM Serving](https://mlsanity.com/paper/2509.12211/) (2025)
- [Designing Large Foundation Models for Efficient Training and Inference: A Survey](https://mlsanity.com/paper/2409.01990/) (2024)
- [PiKV: KV Cache Management System for Mixture of Experts](https://mlsanity.com/paper/2508.06526/) (2025)
- [HSGM: Hierarchical Segment-Graph Memory for Scalable Long-Text Semantics](https://mlsanity.com/paper/2509.18168/) (2025)
- [GraphSnapShot: Caching Local Structure for Fast Graph Learning](https://mlsanity.com/paper/2406.17918/) (2024)
- [SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling](https://mlsanity.com/paper/2508.15190/) (2025)
- [QuickMerge++: Fast Token Merging with Autoregressive Prior](https://mlsanity.com/paper/2508.13204/) (2025)

## Abstract (arXiv)

We propose Cognitive Load Traces (CLTs) as a mid-level interpretability framework for deep models, inspired by Cognitive Load Theory in human cognition. CLTs are defined as symbolic, temporally varying functions that quantify model-internal resource allocation. Formally, we represent CLTs as a three-component stochastic process (ILt, ELt, GLt), corresponding to Intrinsic, Extraneous, and Germane load. Each component is instantiated through measurable proxies such as attention entropy, KV-cache miss ratio, representation dispersion, and decoding stability. We propose both symbolic formulations and visualization methods (load curves, simplex diagrams) that enable interpretable analysis of reasoning dynamics. Experiments on reasoning and planning benchmarks show that CLTs predict error-onset, reveal cognitive strategies, and enable load-guided interventions that improve reasoning efficiency by 15-30% while maintaining accuracy.