# Neuro-SERKET: Development of Integrative Cognitive System through the Composition of Deep Probabilistic Generative Models

arXiv: https://arxiv.org/abs/1910.08918 · HTML: https://mlsanity.com/paper/1910.08918/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2019.

## Built on by (most cited first, 20 of 20)

- [A survey of multimodal deep generative models](https://mlsanity.com/paper/2207.02127/) (2022)
- [World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challenges](https://mlsanity.com/paper/2301.05832/) (2023)
- [Emergent Communication through Metropolis-Hastings Naming Game with Deep Generative Models](https://mlsanity.com/paper/2205.12392/) (2022)
- [Multiagent Multimodal Categorization for Symbol Emergence: Emergent Communication via Interpersonal Cross-modal Inference](https://mlsanity.com/paper/2109.07194/) (2021)
- [Recursive Metropolis-Hastings Naming Game: Symbol Emergence in a Multi-agent System based on Probabilistic Generative Models](https://mlsanity.com/paper/2305.19761/) (2023)
- [The whole brain architecture approach: Accelerating the development of artificial general intelligence by referring to the brain](https://mlsanity.com/paper/2103.06123/) (2021)
- [Hippocampal formation-inspired probabilistic generative model](https://mlsanity.com/paper/2103.07356/) (2021)
- [Metropolis-Hastings algorithm in joint-attention naming game: Experimental semiotics study](https://mlsanity.com/paper/2305.19936/) (2023)
- [Constructive Approach to Bidirectional Influence between Qualia Structure and Language Emergence](https://mlsanity.com/paper/2409.09413/) (2024)
- [A Whole Brain Probabilistic Generative Model: Toward Realizing Cognitive Architectures for Developmental Robots](https://mlsanity.com/paper/2103.08183/) (2021)
- [Toward an Idiomatic Framework for Cognitive Robotics](https://mlsanity.com/paper/2111.13027/) (2021)
- [Symbol Emergence as Inter-personal Categorization with Head-to-head Latent Word](https://mlsanity.com/paper/2205.15027/) (2022)
- [Hierarchical Bayesian Model for the Transfer of Knowledge on Spatial Concepts based on Multimodal Information](https://mlsanity.com/paper/2103.06442/) (2021)
- [Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues](https://mlsanity.com/paper/2201.06786/) (2022)
- [Hierarchical Path-planning from Speech Instructions with Spatial Concept-based Topometric Semantic Mapping](https://mlsanity.com/paper/2203.10820/) (2022)
- [Generative Model-Driven Synthetic Training Image Generation: An Approach to Cognition in Rail Defect Detection](https://mlsanity.com/paper/2401.00393/) (2023)
- [Pixyz: a Python library for developing deep generative models](https://mlsanity.com/paper/2107.13109/) (2021)
- [Brain-inspired probabilistic generative model for double articulation analysis of spoken language](https://mlsanity.com/paper/2207.02457/) (2022)
- [Symbol emergence as interpersonal cross-situational learning: the emergence of lexical knowledge with combinatoriality](https://mlsanity.com/paper/2306.15837/) (2023)
- [Unsupervised Work Behavior Pattern Extraction Based on Hierarchical Probabilistic Model](https://mlsanity.com/paper/2405.09838/) (2024)

## Abstract (arXiv)

This paper describes a framework for the development of an integrative cognitive system based on probabilistic generative models (PGMs) called Neuro-SERKET. Neuro-SERKET is an extension of SERKET, which can compose elemental PGMs developed in a distributed manner and provide a scheme that allows the composed PGMs to learn throughout the system in an unsupervised way. In addition to the head-to-tail connection supported by SERKET, Neuro-SERKET supports tail-to-tail and head-to-head connections, as well as neural network-based modules, i.e., deep generative models. As an example of a Neuro-SERKET application, an integrative model was developed by composing a variational autoencoder (VAE), a Gaussian mixture model (GMM), latent Dirichlet allocation (LDA), and automatic speech recognition (ASR). The model is called VAE+GMM+LDA+ASR. The performance of VAE+GMM+LDA+ASR and the validity of Neuro-SERKET were demonstrated through a multimodal categorization task using image data and a speech signal of numerical digits.