# Oracle Guided Image Synthesis with Relative Queries

arXiv: https://arxiv.org/abs/2204.14189 · HTML: https://mlsanity.com/paper/2204.14189/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 28 Apr 2022.

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Controlling generative models with continuous factors of variations](https://mlsanity.com/paper/2001.10238/) (2020)
- [Disentangling Factors of Variation Using Few Labels](https://mlsanity.com/paper/1905.01258/) (2019)
- [Morpho-MNIST: Quantitative Assessment and Diagnostics for Representation Learning](https://mlsanity.com/paper/1809.10780/) (2018)
- [Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders](https://mlsanity.com/paper/1804.10469/) (2018)
- [Bayesian Triplet Loss: Uncertainty Quantification in Image Retrieval](https://mlsanity.com/paper/2011.12663/) (2020)
- [Bayesian representation learning with oracle constraints](https://mlsanity.com/paper/1506.05011/) (2015)
- [Active embedding search via noisy paired comparisons](https://mlsanity.com/paper/1905.04363/) (2019)

## Abstract (arXiv)

Isolating and controlling specific features in the outputs of generative models in a user-friendly way is a difficult and open-ended problem. We develop techniques that allow an oracle user to generate an image they are envisioning in their head by answering a sequence of relative queries of the form textit"do you prefer image a or image b?" Our framework consists of a Conditional VAE that uses the collected relative queries to partition the latent space into preference-relevant features and non-preference-relevant features. We then use the user's responses to relative queries to determine the preference-relevant features that correspond to their envisioned output image. Additionally, we develop techniques for modeling the uncertainty in images' predicted preference-relevant features, allowing our framework to generalize to scenarios in which the relative query training set contains noise.