# Controlling the Latent Space of GANs through Reinforcement Learning: A Case Study on Task-based Image-to-Image Translation

arXiv: https://arxiv.org/abs/2307.13978 · HTML: https://mlsanity.com/paper/2307.13978/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 16 of 16)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution](https://mlsanity.com/paper/1908.06382/) (2019)
- [STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing](https://mlsanity.com/paper/1904.09709/) (2019)
- [RL-GAN-Net: A Reinforcement Learning Agent Controlled GAN Network for Real-Time Point Cloud Shape Completion](https://mlsanity.com/paper/1904.12304/) (2019)
- [WarpGAN: Automatic Caricature Generation](https://mlsanity.com/paper/1811.10100/) (2018)
- [CAFE-GAN: Arbitrary Face Attribute Editing with Complementary Attention Feature](https://mlsanity.com/paper/2011.11900/) (2020)
- [PA-GAN: Progressive Attention Generative Adversarial Network for Facial Attribute Editing](https://mlsanity.com/paper/2007.05892/) (2020)
- [GeneGAN: Learning Object Transfiguration and Attribute Subspace from Unpaired Data](https://mlsanity.com/paper/1705.04932/) (2017)
- [Biosignal Generation and Latent Variable Analysis with Recurrent Generative Adversarial Networks](https://mlsanity.com/paper/1905.07136/) (2019)
- [Interpreting the Latent Space of Generative Adversarial Networks using Supervised Learning](https://mlsanity.com/paper/2102.12139/) (2021)

## Built on by (most cited first, 3 of 3)

- [CCFace: Classification Consistency for Low-Resolution Face Recognition](https://mlsanity.com/paper/2308.09230/) (2023)
- [Leveraging Ensemble-Based Semi-Supervised Learning for Illicit Account Detection in Ethereum DeFi Transactions](https://mlsanity.com/paper/2412.02408/) (2024)
- [Deep Boosting Multi-Modal Ensemble Face Recognition with Sample-Level Weighting](https://mlsanity.com/paper/2308.09234/) (2023)

## Abstract (arXiv)

Generative Adversarial Networks (GAN) have emerged as a formidable AI tool to generate realistic outputs based on training datasets. However, the challenge of exerting control over the generation process of GANs remains a significant hurdle. In this paper, we propose a novel methodology to address this issue by integrating a reinforcement learning (RL) agent with a latent-space GAN (l-GAN), thereby facilitating the generation of desired outputs. More specifically, we have developed an actor-critic RL agent with a meticulously designed reward policy, enabling it to acquire proficiency in navigating the latent space of the l-GAN and generating outputs based on specified tasks. To substantiate the efficacy of our approach, we have conducted a series of experiments employing the MNIST dataset, including arithmetic addition as an illustrative task. The outcomes of these experiments serve to validate our methodology. Our pioneering integration of an RL agent with a GAN model represents a novel advancement, holding great potential for enhancing generative networks in the future.