Emergent Compositional Communication for Latent World Properties
Not on the map. Why?
Abstract (arXiv)
Can multi-agent communication pressure extract discrete, compositional representations of invisible physical properties from frozen video features? We show that agents communicating through a Gumbel-Softmax bottleneck with iterated learning develop positionally disentangled protocols for latent properties (elasticity, friction, mass ratio) without property labels or supervision on message structure. With 4 agents, 100% of 80 seeds converge to near-perfect compositionality (PosDis=0.999, holdout 98.3%). Controls confirm multi-agent structure -- not bandwidth or temporal coverage -- drives this effect. Causal intervention shows surgical property disruption (~15% drop on targeted property, <3% on others). A controlled backbone comparison reveals that the perceptual prior determines what is communicable: DINOv2 dominates on spatially-visible ramp physics (98.3% vs 95.1%), while V-JEPA 2 dominates on dynamics-only collision physics (87.4% vs 77.7%, d=2.74). Scale-matched (d=3.37) and frame-matched (d=6.53) controls attribute this gap entirely to video-native pretraining. The frozen protocol supports action-conditioned planning (91.5%) with counterfactual velocity reasoning (r=0.780). Validation on Physics 101 real camera footage confirms 85.6% mass-comparison accuracy on unseen objects, temporal dynamics contributing +11.2% beyond static appearance, agent-scaling compositionality replicating at 90% for 4 agents, and causal intervention extending to real video (d=1.87, p=0.022).
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
23 of 33 references.
References · 23
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2022Kubric: A scalable dataset generatorcited by 381
- 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
- 2018TarMAC: Targeted Multi-Agent Communicationcited by 208
- 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
- 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 170
- 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 153
- 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 125
- 2020Compositionality and Generalization in Emergent Languagescited by 103
- 2017Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialogcited by 86
11 more
- 2019Measuring Compositionality in Representation Learningcited by 84
- 2019CoPhy: Counterfactual Learning of Physical Dynamicscited by 81
- 2020Compositional Languages Emerge in a Neural Iterated Learning Modelcited by 75
- 2018IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoningcited by 73
- 2025Intuitive physics understanding emerges from self-supervised pretraining on natural videoscited by 70
- 2019Ease-of-Teaching and Language Structure from Emergent Communicationcited by 65
- 2018Compositional Obverter Communication Learning From Raw Visual Inputcited by 30
- 2022On the role of population heterogeneity in emergent communicationcited by 24
- 2020"LazImpa": Lazy and Impatient neural agents learn to communicate efficientlycited by 23
- 2018Interpretable Intuitive Physics Modelcited by 19
- 2020Emergent Language Generalization and Acquisition Speed are not tied to Compositionalitycited by 17
Built on by
Citing papers per year: 2026: 1.
1 paper.