mlsanityData to 7 Oct 2026

WMF-AM: Probing LLM Working Memory via Depth-Parameterized Cumulative State Tracking

arXiv 2603.27343 ↗First posted 28 Mar 2026cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Existing large language models (LLMs) evaluations use fixed-difficulty benchmarks that cannot adapt as models improve, and rarely isolate specific cognitive processes. We introduce Working Memory Fidelity-Active Manipulation (WMF-AM), a probe of cumulative state tracking, the ability to maintain and update intermediate results across K sequential operations within a single query, without a scratchpad. Unlike multi-step agent benchmarks that stress task orchestration, WMF-AM isolates within-pass cumulative load by parameterizing depth K. The core probe uses arithmetic accumulation on 28 models from 12 families (0.5B to frontier); a matched non-arithmetic extension (permissions, schedules, inventories) confirms the design generalizes beyond arithmetic. Three construct-isolation ablations confirm that cumulative load, not arithmetic skill or entity tracking, drives difficulty. We release WMF-AM as a lightweight, recalibratable diagnostic for characterizing where models degrade under cumulative load. Code and data can be accessed at https://github.com/dengzhe-hou/WMF-AM

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 46 of its 63 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

46 of 63 references.

References · 46

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  11. 2024Qwen2.5 Technical Reportcited by 4,713
  12. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
34 more
  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2023Let's Verify Step by Stepcited by 3,598
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  8. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  9. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  10. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  11. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  12. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  13. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  14. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  15. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  16. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  17. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  18. 2019Measurement and Fairnesscited by 299
  19. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  20. 2023Entity Tracking in Language Modelscited by 44
  21. 2025(How) Do Language Models Track State?cited by 31
  22. 2026AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agentscited by 28
  23. 2024Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalizationcited by 24
  24. 2025Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Lengthcited by 18
  25. 2025Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuitscited by 12
  26. 2025Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoningcited by 11
  27. 2025Minerva: A Programmable Memory Test Benchmark for Language Modelscited by 9
  28. 2026M3-BENCH: Process-Aware Evaluation of LLM Agents' Social Behaviors in Mixed-Motive Gamescited by 8
  29. 2025Strong Memory, Weak Control: An Empirical Study of Executive Functioning in LLMscited by 6
  30. 2025Cognitive Load Limits in Large Language Models: Benchmarking Multi-Hop Reasoningcited by 5
  31. 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbationscited by 5
  32. 2026Intelligence Degradation in Long-Context LLMs: Critical Threshold Determination via Natural Length Distribution Analysiscited by 4
  33. 2026A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilitiescited by 4
  34. 2025Exploring State Tracking Capabilities of Large Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026World-Model Collapse as a Phase Transitioncited by 1
  2. 2026PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoningcited by 0
  3. 2026The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scalingcited by 0
  4. 2026CogArena: A Multimethod Evaluation of Cognitive Ability Structure in Large Language Modelscited by 0