CASS: Cross Adversarial Source Separation via Autoencoder

This paper introduces a cross adversarial source separation (CASS) framework via autoencoder, a new model that aims at separating an input signal consisting of a mixture of multiple components into individual components defined via adversarial learning and autoencoder fitting. CASS unifies popular generative networks like auto-encoders (AEs) and generative adversarial networks (GANs) in a single framework. The basic building block that filters the input signal and reconstructs the i-th target component is a pair of deep neural networks ENi and DEi as an encoder for dimension reduction and a decoder for component reconstruction, respectively. The decoder DEi as a generator is enhanced by a discriminator network Di that favors signal structures of the i-th component in the i-th given dataset as guidance through adversarial learning. In contrast with existing practices in AEs which trains each Auto-Encoder independently, or in GANs that share the same generator, we introduce cross adversarial training that emphasizes adversarial relation between any arbitrary network pairs (DEi,Dj), achieving state-of-the-art performance especially when target components share similar data structures.