Cognitive Activation and Chaotic Dynamics in Large Language Models: A Quasi-Lyapunov Analysis of Reasoning Mechanisms
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
The human-like reasoning capabilities exhibited by Large Language Models (LLMs) challenge the traditional neural network theory's understanding of the flexibility of fixed-parameter systems. This paper proposes the "Cognitive Activation" theory, revealing the essence of LLMs' reasoning mechanisms from the perspective of dynamic systems: the model's reasoning ability stems from a chaotic process of dynamic information extraction in the parameter space. By introducing the Quasi-Lyapunov Exponent (QLE), we quantitatively analyze the chaotic characteristics of the model at different layers. Experiments show that the model's information accumulation follows a nonlinear exponential law, and the Multilayer Perceptron (MLP) accounts for a higher proportion in the final output than the attention mechanism. Further experiments indicate that minor initial value perturbations will have a substantial impact on the model's reasoning ability, confirming the theoretical analysis that large language models are chaotic systems. This research provides a chaos theory framework for the interpretability of LLMs' reasoning and reveals potential pathways for balancing creativity and reliability in model design.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 31 of its 43 references that are in the map. First posted on arXiv 15 Mar 2025.
Builds on
31 of 43 references.
References · 31
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2023A Survey of Large Language Modelscited by 2,537
- 2024Qwen2 Technical Reportcited by 2,076
- 2023A Survey on Evaluation of Large Language Modelscited by 1,227
- 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
- 2022Quantifying Memorization Across Neural Language Modelscited by 755
- 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
19 more
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
- 2021Knowledge Neurons in Pretrained Transformerscited by 590
- 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
- 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
- 2017Stable Architectures for Deep Neural Networkscited by 340
- 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
- 2021Counterfactual Memorization in Neural Language Modelscited by 155
- 2022Analyzing Transformers in Embedding Spacecited by 131
- 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chainscited by 115
- 2022Do Large Language Models know what humans know?cited by 60
- 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Modelscited by 51
- 2021What can linearized neural networks actually say about generalization?cited by 49
- 2021SparseBERT: Rethinking the Importance Analysis in Self-attentioncited by 46
- 2024Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chainscited by 39
- 2024Understanding Transformers via N-gram Statisticscited by 23
- 2022How to Dissect a Muppet: The Structure of Transformer Embedding Spacescited by 19
- 2024Intelligence at the Edge of Chaoscited by 13
- 2024Over the Edge of Chaos? Excess Complexity as a Roadblock to Artificial General Intelligencecited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026ChaosBench-Logic: A Benchmark for Logical and Symbolic Reasoning on Chaotic Dynamical Systems
- 2025A Statistical Physics of Language Model Reasoning
- 2026Neural Activation Patterns Across Language Model Architectures: A Comprehensive Analysis of Cognitive Task Performance
- 2026Understanding Large Language Models
- 2026Mean-Field Dynamics of Chain-of-Thought Reasoning in Large Language Models
- 2026The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Models
- 2025Revisiting the UID Hypothesis in LLM Reasoning Traces
- 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches