# Basic syntax from speech: Spontaneous concatenation in unsupervised deep neural networks

arXiv: https://arxiv.org/abs/2305.01626 · HTML: https://mlsanity.com/paper/2305.01626/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 67 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 15 of 15)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Generative Spoken Language Modeling from Raw Audio](https://mlsanity.com/paper/2102.01192/) (2021)
- [Adversarial Audio Synthesis](https://mlsanity.com/paper/1802.04208/) (2018)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Measuring Compositionality in Representation Learning](https://mlsanity.com/paper/1902.07181/) (2019)
- [Emergent Communication: Generalization and Overfitting in Lewis Games](https://mlsanity.com/paper/2209.15342/) (2022)
- [CiwGAN and fiwGAN: Encoding information in acoustic data to model lexical learning with Generative Adversarial Networks](https://mlsanity.com/paper/2006.02951/) (2020)
- [Generative Adversarial Phonology: Modeling unsupervised phonetic and phonological learning with neural networks](https://mlsanity.com/paper/2006.03965/) (2020)
- [Identity-Based Patterns in Deep Convolutional Networks: Generative Adversarial Phonology and Reduplication](https://mlsanity.com/paper/2009.06110/) (2020)
- [Local and non-local dependency learning and emergence of rule-like representations in speech data by Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/2009.12711/) (2020)
- [Interpreting intermediate convolutional layers of generative CNNs trained on waveforms](https://mlsanity.com/paper/2104.09489/) (2021)
- [Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no direct access to speech data](https://mlsanity.com/paper/2203.11476/) (2022)
- [Interpreting intermediate convolutional layers in unsupervised acoustic word classification](https://mlsanity.com/paper/2110.02375/) (2021)

## Built on by (most cited first, 6 of 6)

- [Language Models as Models of Language](https://mlsanity.com/paper/2408.07144/) (2024)
- [A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models](https://mlsanity.com/paper/2311.00445/) (2023)
- [Linguistic Structure from a Bottleneck on Sequential Information Processing](https://mlsanity.com/paper/2405.12109/) (2024)
- [Disentanglement in a GAN for Unconditional Speech Synthesis](https://mlsanity.com/paper/2307.01673/) (2023)
- [Unsupervised Learning and Representation of Mandarin Tonal Categories by a Generative CNN](https://mlsanity.com/paper/2509.17859/) (2025)
- [A Technique for Isolating Lexically-Independent Phonetic Dependencies in Generative CNNs](https://mlsanity.com/paper/2506.09218/) (2025)

## Abstract (arXiv)

Computational models of syntax are predominantly text-based. Here we propose that the most basic first step in the evolution of syntax can be modeled directly from raw speech in a fully unsupervised way. We focus on one of the most ubiquitous and elementary suboperations of syntax -- concatenation. We introduce textitspontaneous concatenation: a phenomenon where a ciwGAN/fiwGAN models (based on convolutional neural networks) trained on acoustic recordings of individual words start generating outputs with two or even three words concatenated without ever accessing data with multiple words in the training data. We replicate this finding in several independently trained models with different hyperparameters and training data. Additionally, networks trained on two words learn to embed words into novel unobserved word combinations. We also show that the concatenated outputs contain precursors to compositionality. To our knowledge, this is a previously unreported property of CNNs trained in the ciwGAN/fiwGAN setting on raw speech and has implications both for our understanding of how these architectures learn as well as for modeling syntax and its evolution in the brain from raw acoustic inputs. We also propose and formalize a neural mechanism called textitdisinhibition that outlines a possible artificial and biological neural pathway towards concatenation and compositionality and suggests our modeling is useful for generating testable predictions for biological and artificial neural processing of spoken language.