mlsanityData to 14 Oct 2026

Transcript-Managed Transformers: Monotone Multi-Agent Collapse and Universality with Two Pop-Enabled Transcripts

arXiv 2607.29496 ↗First posted 31 Jul 2026cs.LG cs.FL cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

We study transcript management for fixed, finite-precision causal Transformers. A transcript is partitioned into channels of bounded blocks. Each transition consults a fixed visible suffix and may append one block, leaving the model, weights, and token protocol unchanged. The operation Pc:=PopContext(c) deletes the newest block on channel c and exposes its predecessor. We model the layer by the Transcript-Managed Transducer TMTnk: one finite controller, k channels, and per-round actions from stay, push, and pop under a caller-driven status map. Fixed visible windows encode as finite symbols. The pop-free Restricted Transcript-Managed Transducer RTMTnk is the standard append-only layer and, for every fixed k, realizes exactly the deterministic finite-state transductions. The same holds for every fixed finite agent population under a monotone protocol that appends, routes, and copies visible blocks. Admitting Pcc=1k restores pop. Newest-first, a pop-enabled channel is a stack; compiling to the Hopcroft--Ullman presentation transfers the classical hierarchy: DCFL for k=1 and RE for every k≥2. Orchestrated one-channel agents match one controller with k channels, so two pop-enabled transcripts---in one agent or two---suffice for universality. Simulation costs and invariance to fixed block size and visible radius are stated. The bounds fix precision, alphabets, blocks, visibility, controller state, and population; growing exact context, hidden-block access, writable stores, and unbounded Spawn add further state.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

35 of 53 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  7. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  8. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  9. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  10. 2018Universal Transformerscited by 647
  11. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 327
  12. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 294
23 more
  1. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 288
  2. 2022Neural Networks and the Chomsky Hierarchycited by 217
  3. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 193
  4. 2021Saturated Transformers are Constant-Depth Threshold Circuitscited by 139
  5. 2023What Formal Languages Can Transformers Express? A Surveycited by 124
  6. 2024Transformers are Multi-State RNNscited by 120
  7. 2022Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexitycited by 93
  8. 2020On the Computational Power of Transformers and its Implications in Sequence Modelingcited by 85
  9. 2015Inferring Algorithmic Patterns with Stack-Augmented Recurrent Netscited by 76
  10. 2022Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functionscited by 70
  11. 2017Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamplescited by 66
  12. 2015Learning to Transduce with Unbounded Memorycited by 54
  13. 2024Base of RoPE Bounds Context Lengthcited by 52
  14. 2020Do Transformers Need Deep Long-Range Memorycited by 24
  15. 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Modelscited by 19
  16. 2015A Survey and Classification of Controlled Natural Languagescited by 18
  17. 2023Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patternscited by 17
  18. 2020Learning Context-Free Languages with Nondeterministic Stack RNNscited by 15
  19. 2025Benefits and Limitations of Communication in Multi-Agent Reasoningcited by 7
  20. 2026Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformerscited by 7
  21. 2026When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agentscited by 7
  22. 2020On Systematically Building a Controlled Natural Language for Functional Requirementscited by 5
  23. 2025Efficient Turing Machine Simulation with Transformerscited by 4

Built on by

Not cited yet.