What You Can't See Is What You Learn: Slot-Selective Evidence Masking Favors Compositional Generalization in Shared-Genome Language-Model Societies
Not on the map. Why?
Abstract (arXiv)
Multi-module neural systems often expose every module to the full input. We test whether a slot-selective evidence-masking regime -- restricting each module to its own evidence span -- changes which solutions gradient-based training discovers. Four-cell societies share one frozen pretrained language model and one low-rank adapter, communicating only through two model-width continuous vectors in a fixed relay. On a prospectively sealed natural-language function-composition task, we train ten matched restricted/global pairs identical except for the attention mask. Restricted-visibility societies outperform their globally visible twins by at least 20 percentage points at both depths in 9 of 10 pairs, with median paired advantages of 0.7648 and 0.6050. Cutting communication reduces every restricted society to chance, and in a post hoc collision-stratified analysis the depth-three advantage remains 0.558 on programs whose complete affine map never appeared in training. In six post hoc-selected restricted societies, packet interventions on correctly answered held-out episodes are consistent with approximately value-indexed relay states; the sole high-performing global model also requires communication, but its same-value packets are not interchangeable across episodes. Thus restricted visibility is not necessary for composition. Under the tested seeds, streams, task world, and training budget, the masking regime strongly shifted which solutions training discovered: a post hoc mask crossover finds both arms mask-native. Because the restricted mask both blocks foreign evidence and implicitly identifies each cell's assigned slot, attribution to evidence visibility alone awaits a role-marked control. The preregistered battery nevertheless formally fails because restricted-arm median depth-three accuracy is 0.6988, below the 0.70 floor; an earlier qualification cohort yielded 0/10 complete passes.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 20 Aug 2026.
Builds on
27 of 35 references.
References · 27
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
- 2021Causal Abstractions of Neural Networkscited by 422
- 2019Recurrent Independent Mechanismscited by 219
- 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
- 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
- 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
- 2020Compositionality and Generalization in Emergent Languagescited by 103
- 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 85
15 more
- 2019Capacity, Bandwidth, and Compositionality in Emergent Language Learningcited by 30
- 2022Emergent Communication: Generalization and Overfitting in Lewis Gamescited by 28
- 2021How Modular Should Neural Module Networks Be for Systematic Generalization?cited by 15
- 2025Scaling can lead to compositional generalizationcited by 7
- 2021Dynamics of specialization in neural modules under resource constraintscited by 6
- 2026Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLMcited by 5
- 2026StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systemscited by 5
- 2026AC-VLA: Robust Out-of-Distribution Action Execution via Compositional Learningcited by 3
- 2026When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMscited by 3
- 2026XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communicationcited by 3
- 2026Input Pathways Shape Few-Shot, Not Zero-Shot, Binding in Tiny Transformers: A Fully-Enumerable Studycited by 2
- 2026Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futurescited by 2
- 2026Emergent Compositional Communication for Latent World Propertiescited by 1
- 2026Making Transformers Logicalcited by 1
- 2026Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledgecited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.