# Bayesian representation learning with oracle constraints

arXiv: https://arxiv.org/abs/1506.05011 · HTML: https://mlsanity.com/paper/1506.05011/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2015.

## Built on by (most cited first, 16 of 16)

- [StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models](https://mlsanity.com/paper/2308.07863/) (2023)
- [Self-Supervised Learning Disentangled Group Representation as Feature](https://mlsanity.com/paper/2110.15255/) (2021)
- [Disentangling Hate in Online Memes](https://mlsanity.com/paper/2108.06207/) (2021)
- [Modelling Cellular Perturbations with the Sparse Additive Mechanism Shift Variational Autoencoder](https://mlsanity.com/paper/2311.02794/) (2023)
- [Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation](https://mlsanity.com/paper/2108.07668/) (2021)
- [DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations](https://mlsanity.com/paper/2203.02013/) (2022)
- [Symmetry-Based Representations for Artificial and Biological General Intelligence](https://mlsanity.com/paper/2203.09250/) (2022)
- [InfoNCE is variational inference in a recognition parameterised model](https://mlsanity.com/paper/2107.02495/) (2021)
- [Distortion-Disentangled Contrastive Learning](https://mlsanity.com/paper/2303.05066/) (2023)
- [Evaluation of Correctness in Unsupervised Many-to-Many Image Translation](https://mlsanity.com/paper/2103.15727/) (2021)
- [Active metric learning and classification using similarity queries](https://mlsanity.com/paper/2202.01953/) (2022)
- [Multi-modal data generation with a deep metric variational autoencoder](https://mlsanity.com/paper/2202.03434/) (2022)
- [Boosting Few-Shot Learning with Disentangled Self-Supervised Learning and Meta-Learning for Medical Image Classification](https://mlsanity.com/paper/2403.17530/) (2024)
- [SCAdapter: Content-Style Disentanglement for Diffusion Style Transfer](https://mlsanity.com/paper/2512.12963/) (2025)
- [Semi-Supervised Few-Shot Classification with Deep Invertible Hybrid Models](https://mlsanity.com/paper/2105.10644/) (2021)
- [Oracle Guided Image Synthesis with Relative Queries](https://mlsanity.com/paper/2204.14189/) (2022)

## Abstract (arXiv)

Representation learning systems typically rely on massive amounts of labeled data in order to be trained to high accuracy. Recently, high-dimensional parametric models like neural networks have succeeded in building rich representations using either compressive, reconstructive or supervised criteria. However, the semantic structure inherent in observations is oftentimes lost in the process. Human perception excels at understanding semantics but cannot always be expressed in terms of labels. Thus, oracles or human-in-the-loop systems, for example crowdsourcing, are often employed to generate similarity constraints using an implicit similarity function encoded in human perception. In this work we propose to combine generative unsupervised feature learning with a probabilistic treatment of oracle information like triplets in order to transfer implicit privileged oracle knowledge into explicit nonlinear Bayesian latent factor models of the observations. We use a fast variational algorithm to learn the joint model and demonstrate applicability to a well-known image dataset. We show how implicit triplet information can provide rich information to learn representations that outperform previous metric learning approaches as well as generative models without this side-information in a variety of predictive tasks. In addition, we illustrate that the proposed approach compartmentalizes the latent spaces semantically which allows interpretation of the latent variables.