mlsanityData to 14 Oct 2026

The Coordinate System Problem in Persistent Structural Memory for Neural Architectures

arXiv 2603.22858 ↗First posted 24 Mar 2026cs.LG cs.AI cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce the Dual-View Pheromone Pathway Network (DPPN), an architecture that routes sparse attention through a persistent pheromone field over latent slot transitions, and use it to discover two independent requirements for persistent structural memory in neural networks. Through five progressively refined experiments using up to 10 seeds per condition across 5 model variants and 4 transfer targets, we identify a core principle: persistent memory requires a stable coordinate system, and any coordinate system learned jointly with the model is inherently unstable. We characterize three obstacles -- pheromone saturation, surface-structure entanglement, and coordinate incompatibility -- and show that neither contrastive updates, multi-source distillation, Hungarian alignment, nor semantic decomposition resolves the instability when embeddings are learned from scratch. Fixed random Fourier features provide extrinsic coordinates that are stable, structure-blind, and informative, but coordinate stability alone is insufficient: routing-bias pheromone does not transfer (10 seeds, p>0.05). DPPN outperforms transformer and random sparse baselines for within-task learning (AULC 0.700 vs 0.680 vs 0.670). Replacing routing bias with learning-rate modulation eliminates negative transfer: warm pheromone as a learning-rate prior achieves +0.003 on same-family tasks (17 seeds, p<0.05) while never reducing performance. A structure completion function over extrinsic coordinates produces +0.006 same-family bonus beyond regularization, showing the catch-22 between stability and informativeness is partially permeable to learned functions. The contribution is two independent requirements for persistent structural memory: (a) coordinate stability and (b) graceful transfer mechanism.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 48 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

17 of 48 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2020Longformer: The Long-Document Transformercited by 3,413
  6. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  9. 2019Invariant Risk Minimizationcited by 1,854
  10. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  11. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  12. 2016Progressive Neural Networkscited by 1,361
5 more
  1. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  2. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  3. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  4. 2014Neural Turing Machinescited by 644
  5. 2014Memory Networkscited by 363

Built on by

Not cited yet.