# Identity-Based Patterns in Deep Convolutional Networks: Generative Adversarial Phonology and Reduplication

arXiv: https://arxiv.org/abs/2009.06110 · HTML: https://mlsanity.com/paper/2009.06110/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2020.

## Built on by (most cited first, 9 of 9)

- [Interpreting intermediate convolutional layers of generative CNNs trained on waveforms](https://mlsanity.com/paper/2104.09489/) (2021)
- [Basic syntax from speech: Spontaneous concatenation in unsupervised deep neural networks](https://mlsanity.com/paper/2305.01626/) (2023)
- [Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no direct access to speech data](https://mlsanity.com/paper/2203.11476/) (2022)
- [Approaching an unknown communication system by latent space exploration and causal inference](https://mlsanity.com/paper/2303.10931/) (2023)
- [Exploring How Generative Adversarial Networks Learn Phonological Representations](https://mlsanity.com/paper/2305.12501/) (2023)
- [Do RNN States Encode Abstract Phonological Processes?](https://mlsanity.com/paper/2104.00789/) (2021)
- [CiwaGAN: Articulatory information exchange](https://mlsanity.com/paper/2309.07861/) (2023)
- [Deciphering Assamese Vowel Harmony with Featural InfoWaveGAN](https://mlsanity.com/paper/2407.06547/) (2024)
- [A Technique for Isolating Lexically-Independent Phonetic Dependencies in Generative CNNs](https://mlsanity.com/paper/2506.09218/) (2025)

## Abstract (arXiv)

This paper models unsupervised learning of an identity-based pattern (or copying) in speech called reduplication from raw continuous data with deep convolutional neural networks. We use the ciwGAN architecture Beguvs (2021a; arXiv:2006.02951) in which learning of meaningful representations in speech emerges from a requirement that the CNNs generate informative data. We propose a technique to wug-test CNNs trained on speech and, based on four generative tests, argue that the network learns to represent an identity-based pattern in its latent space. By manipulating only two categorical variables in the latent space, we can actively turn an unreduplicated form into a reduplicated form with no other substantial changes to the output in the majority of cases. We also argue that the network extends the identity-based pattern to unobserved data. Exploration of how meaningful representations of identity-based patterns emerge in CNNs and how the latent space variables outside of the training range correlate with identity-based patterns in the output has general implications for neural network interpretability.