# Learning Independent Features with Adversarial Nets for Non-linear ICA

arXiv: https://arxiv.org/abs/1710.05050 · HTML: https://mlsanity.com/paper/1710.05050/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2017.

## Built on by (most cited first, 30 of 33)

- [AID-Purifier: A Light Auxiliary Network for Boosting Adversarial Defense](https://mlsanity.com/paper/2107.06456/) (2021)
- [Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations](https://mlsanity.com/paper/2209.14905/) (2022)
- [DRIM: Learning Disentangled Representations from Incomplete Multimodal Healthcare Data](https://mlsanity.com/paper/2409.17055/) (2024)
- [A Normative and Biologically Plausible Algorithm for Independent Component Analysis](https://mlsanity.com/paper/2111.08858/) (2021)
- [Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer](https://mlsanity.com/paper/2208.12527/) (2022)
- [Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection](https://mlsanity.com/paper/2106.12382/) (2021)
- [CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervision](https://mlsanity.com/paper/2112.07133/) (2021)
- [Causal Contrastive Learning for Counterfactual Regression Over Time](https://mlsanity.com/paper/2406.00535/) (2024)
- [Estimating and Maximizing Mutual Information for Knowledge Distillation](https://mlsanity.com/paper/2110.15946/) (2021)
- [PluGeN: Multi-Label Conditional Generation From Pre-Trained Models](https://mlsanity.com/paper/2109.09011/) (2021)
- [The Hidden Link Between RLHF and Contrastive Learning](https://mlsanity.com/paper/2506.22578/) (2025)
- [On the relationship between disentanglement and multi-task learning](https://mlsanity.com/paper/2110.03498/) (2021)
- [Representation Learning for Conversational Data using Discourse Mutual Information Maximization](https://mlsanity.com/paper/2112.05787/) (2021)
- [A robust estimator of mutual information for deep learning interpretability](https://mlsanity.com/paper/2211.00024/) (2022)
- [An Independence-promoting Loss for Music Generation with Language Models](https://mlsanity.com/paper/2406.02315/) (2024)
- [Discovery of Single Independent Latent Variable](https://mlsanity.com/paper/2110.05887/) (2021)
- [Deep Deterministic Independent Component Analysis for Hyperspectral Unmixing](https://mlsanity.com/paper/2202.02951/) (2022)
- [Disentangling Autoencoders (DAE)](https://mlsanity.com/paper/2202.09926/) (2022)
- [Enhancing the "Immunity" of Mixture-of-Experts Networks for Adversarial Defense](https://mlsanity.com/paper/2402.18787/) (2024)
- [Single-preparation unsupervised quantum machine learning: concepts and applications](https://mlsanity.com/paper/2101.01442/) (2021)
- [Non-parametric Probabilistic Time Series Forecasting via Innovations Representation](https://mlsanity.com/paper/2306.03782/) (2023)
- [Half-AVAE: Adversarial-Enhanced Factorized and Structured Encoder-Free VAE for Underdetermined Independent Component Analysis](https://mlsanity.com/paper/2506.07011/) (2025)
- [Structured Kernel Regression VAE: A Computationally Efficient Surrogate for GP-VAEs in ICA](https://mlsanity.com/paper/2508.09721/) (2025)
- [Novelty Detection in Time Series via Weak Innovations Representation: A Deep Learning Approach](https://mlsanity.com/paper/2210.13358/) (2022)
- [Blind Source Separation of Single-Channel Mixtures via Multi-Encoder Autoencoders](https://mlsanity.com/paper/2309.07138/) (2023)
- [Adversarial Dependence Minimization](https://mlsanity.com/paper/2502.03227/) (2025)
- [Refining the Information Bottleneck via Adversarial Information Separation](https://mlsanity.com/paper/2602.06549/) (2026)
- [Weakly Supervised Recovery of Semantic Attributes](https://mlsanity.com/paper/2103.11888/) (2021)
- [Visual processing in context of reinforcement learning](https://mlsanity.com/paper/2208.12525/) (2022)
- [Semi-Blind Source Separation with Learned Constraints](https://mlsanity.com/paper/2209.13585/) (2022)

## Abstract (arXiv)

Reliable measures of statistical dependence could be useful tools for learning independent features and performing tasks like source separation using Independent Component Analysis (ICA). Unfortunately, many of such measures, like the mutual information, are hard to estimate and optimize directly. We propose to learn independent features with adversarial objectives which optimize such measures implicitly. These objectives compare samples from the joint distribution and the product of the marginals without the need to compute any probability densities. We also propose two methods for obtaining samples from the product of the marginals using either a simple resampling trick or a separate parametric distribution. Our experiments show that this strategy can easily be applied to different types of model architectures and solve both linear and non-linear ICA problems.