# Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling

arXiv: https://arxiv.org/abs/1709.08868 · HTML: https://mlsanity.com/paper/1709.08868/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2017.

## Built on by (most cited first, 30 of 32)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds](https://mlsanity.com/paper/2109.00179/) (2021)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [EM Distillation for One-step Diffusion Models](https://mlsanity.com/paper/2405.16852/) (2024)
- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model](https://mlsanity.com/paper/2205.06924/) (2022)
- [TabPFGen -- Tabular Data Generation with TabPFN](https://mlsanity.com/paper/2406.05216/) (2024)
- [Diffusive Gibbs Sampling](https://mlsanity.com/paper/2402.03008/) (2024)
- [Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood](https://mlsanity.com/paper/2309.05153/) (2023)
- [EGC: Image Generation and Classification via a Diffusion Energy-Based Model](https://mlsanity.com/paper/2304.02012/) (2023)
- [Learning Probabilistic Models from Generator Latent Spaces with Hat EBM](https://mlsanity.com/paper/2210.16486/) (2022)
- [Sampling Multimodal Distributions with the Vanilla Score: Benefits of Data-Based Initialization](https://mlsanity.com/paper/2310.01762/) (2023)
- [Multilevel CNNs for Parametric PDEs](https://mlsanity.com/paper/2304.00388/) (2023)
- [Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation](https://mlsanity.com/paper/2103.04285/) (2021)
- [Energy-Based Generative Cooperative Saliency Prediction](https://mlsanity.com/paper/2106.13389/) (2021)
- [Moment Matching Denoising Gibbs Sampling](https://mlsanity.com/paper/2305.11650/) (2023)
- [Gradient-Guided Importance Sampling for Learning Binary Energy-Based Models](https://mlsanity.com/paper/2210.05782/) (2022)
- [Learning Unnormalized Statistical Models via Compositional Optimization](https://mlsanity.com/paper/2306.07485/) (2023)
- [EQ-CBM: A Probabilistic Concept Bottleneck with Energy-based Models and Quantized Vectors](https://mlsanity.com/paper/2409.14630/) (2024)
- [Learning Proposals for Practical Energy-Based Regression](https://mlsanity.com/paper/2110.11948/) (2021)
- [An Energy-Based Prior for Generative Saliency](https://mlsanity.com/paper/2204.08803/) (2022)
- [On the Robustness of Langevin Dynamics to Score Function Error](https://mlsanity.com/paper/2603.11319/) (2026)
- [Towards Theoretical Analysis of Transformation Complexity of ReLU DNNs](https://mlsanity.com/paper/2205.01940/) (2022)
- [Information-Theoretic GAN Compression with Variational Energy-based Model](https://mlsanity.com/paper/2303.16050/) (2023)
- [On Discriminative Probabilistic Modeling for Self-Supervised Representation Learning](https://mlsanity.com/paper/2410.09156/) (2024)
- [CoopHash: Cooperative Learning of Multipurpose Descriptor and Contrastive Pair Generator via Variational MCMC Teaching for Supervised Image Hashing](https://mlsanity.com/paper/2210.04288/) (2022)
- [Progressive Energy-Based Cooperative Learning for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/2306.14448/) (2023)
- [EBM Life Cycle: MCMC Strategies for Synthesis, Defense, and Density Modeling](https://mlsanity.com/paper/2205.12243/) (2022)
- [Non-Generative Energy Based Models](https://mlsanity.com/paper/2304.01297/) (2023)

## Abstract (arXiv)

This paper proposes a multi-grid method for learning energy-based generative ConvNet models of images. For each grid, we learn an energy-based probabilistic model where the energy function is defined by a bottom-up convolutional neural network (ConvNet or CNN). Learning such a model requires generating synthesized examples from the model. Within each iteration of our learning algorithm, for each observed training image, we generate synthesized images at multiple grids by initializing the finite-step MCMC sampling from a minimal 1 x 1 version of the training image. The synthesized image at each subsequent grid is obtained by a finite-step MCMC initialized from the synthesized image generated at the previous coarser grid. After obtaining the synthesized examples, the parameters of the models at multiple grids are updated separately and simultaneously based on the differences between synthesized and observed examples. We show that this multi-grid method can learn realistic energy-based generative ConvNet models, and it outperforms the original contrastive divergence (CD) and persistent CD.