mlsanityData to 7 Oct 2026

Rayleigh EigenDirections (REDs): GAN latent space traversals for multidimensional features

arXiv 2201.10423 ↗First posted 25 Jan 2022cs.CV cs.GR cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

We present a method for finding paths in a deep generative model's latent space that can maximally vary one set of image features while holding others constant. Crucially, unlike past traversal approaches, ours can manipulate multidimensional features of an image such as facial identity and pixels within a specified region. Our method is principled and conceptually simple: optimal traversal directions are chosen by maximizing differential changes to one feature set such that changes to another set are negligible. We show that this problem is nearly equivalent to one of Rayleigh quotient maximization, and provide a closed-form solution to it based on solving a generalized eigenvalue equation. We use repeated computations of the corresponding optimal directions, which we call Rayleigh EigenDirections (REDs), to generate appropriately curved paths in latent space. We empirically evaluate our method using StyleGAN2 on two image domains: faces and living rooms. We show that our method is capable of controlling various multidimensional features out of the scope of previous latent space traversal methods: face identity, spatial frequency bands, pixels within a region, and the appearance and position of an object. Our work suggests that a wealth of opportunities lies in the local analysis of the geometry and semantics of latent spaces.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 42 references that are in the map. First posted on arXiv 25 Jan 2022.

Builds on

36 of 42 references.

References · 36

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  4. 2014Conditional Generative Adversarial Netscited by 2,152
  5. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  6. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  7. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  8. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  9. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  10. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  11. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
  12. 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 288
24 more
  1. 2019On the "steerability" of generative adversarial networkscited by 247
  2. 2017AttGAN: Facial Attribute Editing by Only Changing What You Wantcited by 241
  3. 2019GANalyze: Toward Visual Definitions of Cognitive Image Propertiescited by 193
  4. 2017Latent Space Oddity: on the Curvature of Deep Generative Modelscited by 169
  5. 2017CausalGAN: Learning Causal Implicit Generative Models with Adversarial Trainingcited by 152
  6. 2019STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editingcited by 119
  7. 2017Fader Networks: Manipulating Images by Sliding Attributescited by 118
  8. 2019Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesiscited by 106
  9. 2020Controlling generative models with continuous factors of variationscited by 93
  10. 2021GAN-Control: Explicitly Controllable GANscited by 91
  11. 2017The Riemannian Geometry of Deep Generative Modelscited by 81
  12. 2016Deep Feature Interpolation for Image Content Changescited by 80
  13. 2018Towards Open-Set Identity Preserving Face Synthesiscited by 74
  14. 2020Towards causal benchmarking of bias in face analysis algorithmscited by 64
  15. 2020Lifespan Age Transformation Synthesiscited by 60
  16. 2017Metrics for Deep Generative Modelscited by 53
  17. 2017Face Aging With Conditional Generative Adversarial Networkscited by 51
  18. 2017Towards Large-Pose Face Frontalization in the Wildcited by 50
  19. 2018ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributescited by 50
  20. 2021WarpedGANSpace: Finding non-linear RBF paths in GAN latent spacecited by 48
  21. 2019Eigenvalue and Generalized Eigenvalue Problems: Tutorialcited by 45
  22. 2018FaceFeat-GAN: a Two-Stage Approach for Identity-Preserving Face Synthesiscited by 14
  23. 2018Latent Space Non-Linear Statisticscited by 12
  24. 2020Max-Affine Spline Insights into Deep Generative Networkscited by 10

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1.

2 papers.

  1. 2022Orthogonal SVD Covariance Conditioning and Latent Disentanglementcited by 5
  2. 20233D GANs and Latent Space: A comprehensive surveycited by 1