# Rayleigh EigenDirections (REDs): GAN latent space traversals for multidimensional features

arXiv: https://arxiv.org/abs/2201.10423 · HTML: https://mlsanity.com/paper/2201.10423/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 42 references that are in the map.
- First posted on arXiv 25 Jan 2022.

## Builds on (most cited first, 30 of 36)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [GANalyze: Toward Visual Definitions of Cognitive Image Properties](https://mlsanity.com/paper/1906.10112/) (2019)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://mlsanity.com/paper/1709.02023/) (2017)
- [STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing](https://mlsanity.com/paper/1904.09709/) (2019)
- [Fader Networks: Manipulating Images by Sliding Attributes](https://mlsanity.com/paper/1706.00409/) (2017)
- [Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis](https://mlsanity.com/paper/1911.09267/) (2019)
- [Controlling generative models with continuous factors of variations](https://mlsanity.com/paper/2001.10238/) (2020)
- [GAN-Control: Explicitly Controllable GANs](https://mlsanity.com/paper/2101.02477/) (2021)
- [The Riemannian Geometry of Deep Generative Models](https://mlsanity.com/paper/1711.08014/) (2017)
- [Deep Feature Interpolation for Image Content Changes](https://mlsanity.com/paper/1611.05507/) (2016)
- [Towards Open-Set Identity Preserving Face Synthesis](https://mlsanity.com/paper/1803.11182/) (2018)
- [Towards causal benchmarking of bias in face analysis algorithms](https://mlsanity.com/paper/2007.06570/) (2020)
- [Lifespan Age Transformation Synthesis](https://mlsanity.com/paper/2003.09764/) (2020)
- [Metrics for Deep Generative Models](https://mlsanity.com/paper/1711.01204/) (2017)
- [Face Aging With Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1702.01983/) (2017)
- [Towards Large-Pose Face Frontalization in the Wild](https://mlsanity.com/paper/1704.06244/) (2017)

## Built on by (most cited first, 2 of 2)

- [Orthogonal SVD Covariance Conditioning and Latent Disentanglement](https://mlsanity.com/paper/2212.05599/) (2022)
- [3D GANs and Latent Space: A comprehensive survey](https://mlsanity.com/paper/2304.03932/) (2023)

## Abstract (arXiv)

We present a method for finding paths in a deep generative model's latent space that can maximally vary one set of image features while holding others constant. Crucially, unlike past traversal approaches, ours can manipulate multidimensional features of an image such as facial identity and pixels within a specified region. Our method is principled and conceptually simple: optimal traversal directions are chosen by maximizing differential changes to one feature set such that changes to another set are negligible. We show that this problem is nearly equivalent to one of Rayleigh quotient maximization, and provide a closed-form solution to it based on solving a generalized eigenvalue equation. We use repeated computations of the corresponding optimal directions, which we call Rayleigh EigenDirections (REDs), to generate appropriately curved paths in latent space. We empirically evaluate our method using StyleGAN2 on two image domains: faces and living rooms. We show that our method is capable of controlling various multidimensional features out of the scope of previous latent space traversal methods: face identity, spatial frequency bands, pixels within a region, and the appearance and position of an object. Our work suggests that a wealth of opportunities lies in the local analysis of the geometry and semantics of latent spaces.