Inhibitory Cross-Talk Enables Functional Lateralization in Attention-Coupled Latent Memory

We present a memory-augmented transformer in which attention serves simultaneously as a retrieval, consolidation, and write-back operator. The core update, A^top A V W, re-grounds retrieved values into persistent memory slots via the Gram matrix A^top A, providing a principled tripartite projection: observation space → latent memory → supervised transformation. We partition the memory into lateralized left and right banks coupled through a sign-controlled cross-talk matrix Ws, and show that the sign of this coupling is decisive for specialization. Excitatory cross-talk (s=+1) causes bank-dominance collapse: one bank monopolises all inputs and Pct → 0.5, despite lowering task loss. Inhibitory cross-talk (s=-1), motivated by the net inhibitory effect of callosal projections in human cortex, actively suppresses contralateral bank activation and achieves saturated specialization (Dsep = ± 1.00, Pct ≈ 0). On a controlled symbolic benchmark combining an episodic bijection cipher (requiring associative recall) with a strict arithmetic progression (requiring rule extraction), the inhibitory model reduces cipher-domain loss by 124× over the baseline while matching it on the arithmetic domain, confirming that persistent lateralized memory is necessary for episodic recall but not for rule-based prediction.