# Gradient Origin Networks

arXiv: https://arxiv.org/abs/2007.02798 · HTML: https://mlsanity.com/paper/2007.02798/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2020.

## Built on by (most cited first, 10 of 10)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Generative Models as Distributions of Functions](https://mlsanity.com/paper/2102.04776/) (2021)
- [Implicit Neural Representations for Variable Length Human Motion Generation](https://mlsanity.com/paper/2203.13694/) (2022)
- [∞-Diff: Infinite Resolution Diffusion with Subsampled Mollified States](https://mlsanity.com/paper/2303.18242/) (2023)
- [Half-VAE: An Encoder-Free VAE to Bypass Explicit Inverse Mapping](https://mlsanity.com/paper/2409.04140/) (2024)
- [The Deep Generative Decoder: MAP estimation of representations improves modeling of single-cell RNA data](https://mlsanity.com/paper/2110.06672/) (2021)
- [Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation](https://mlsanity.com/paper/2110.05055/) (2021)
- [GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learning](https://mlsanity.com/paper/2312.08744/) (2023)
- [MING: A Functional Approach to Learning Molecular Generative Models](https://mlsanity.com/paper/2410.12522/) (2024)

## Abstract (arXiv)

This paper proposes a new type of generative model that is able to quickly learn a latent representation without an encoder. This is achieved using empirical Bayes to calculate the expectation of the posterior, which is implemented by initialising a latent vector with zeros, then using the gradient of the log-likelihood of the data with respect to this zero vector as new latent points. The approach has similar characteristics to autoencoders, but with a simpler architecture, and is demonstrated in a variational autoencoder equivalent that permits sampling. This also allows implicit representation networks to learn a space of implicit functions without requiring a hypernetwork, retaining their representation advantages across datasets. The experiments show that the proposed method converges faster, with significantly lower reconstruction error than autoencoders, while requiring half the parameters.