# Locally Masked Convolution for Autoregressive Models

arXiv: https://arxiv.org/abs/2006.12486 · HTML: https://mlsanity.com/paper/2006.12486/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2020.

## Built on by (most cited first, 22 of 22)

- [Autoregressive Diffusion Models](https://mlsanity.com/paper/2110.02037/) (2021)
- [ConvMAE: Masked Convolution Meets Masked Autoencoders](https://mlsanity.com/paper/2205.03892/) (2022)
- [PixelSynth: Generating a 3D-Consistent Experience from a Single Image](https://mlsanity.com/paper/2108.05892/) (2021)
- [BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery](https://mlsanity.com/paper/2112.02761/) (2021)
- [Bayesian Flow Networks](https://mlsanity.com/paper/2308.07037/) (2023)
- [Pixel-Inconsistency Modeling for Image Manipulation Localization](https://mlsanity.com/paper/2310.00234/) (2023)
- [Efficient-VDVAE: Less is more](https://mlsanity.com/paper/2203.13751/) (2022)
- [Generative modeling with projected entangled-pair states](https://mlsanity.com/paper/2202.08177/) (2022)
- [Simulating the Real World: A Unified Survey of Multimodal Generative Models](https://mlsanity.com/paper/2503.04641/) (2025)
- [PixelTransformer: Sample Conditioned Signal Generation](https://mlsanity.com/paper/2103.15813/) (2021)
- [Semi-Discrete Normalizing Flows through Differentiable Tessellation](https://mlsanity.com/paper/2203.06832/) (2022)
- [Investigating Topological Order using Recurrent Neural Networks](https://mlsanity.com/paper/2303.11207/) (2023)
- [The DEformer: An Order-Agnostic Distribution Estimating Transformer](https://mlsanity.com/paper/2106.06989/) (2021)
- [Generation of Synthetic Electronic Health Records Using a Federated GAN](https://mlsanity.com/paper/2109.02543/) (2021)
- [Lossless Compression with Latent Variable Models](https://mlsanity.com/paper/2104.10544/) (2021)
- [Flow-based Spatio-Temporal Structured Prediction of Motion Dynamics](https://mlsanity.com/paper/2104.04391/) (2021)
- [AdaCat: Adaptive Categorical Discretization for Autoregressive Models](https://mlsanity.com/paper/2208.02246/) (2022)
- [Large-Scale Auto-Regressive Modeling Of Street Networks](https://mlsanity.com/paper/2209.00281/) (2022)
- [GMConv: Modulating Effective Receptive Fields for Convolutional Kernels](https://mlsanity.com/paper/2302.04544/) (2023)
- [Leveraging progressive model and overfitting for efficient learned image compression](https://mlsanity.com/paper/2210.04112/) (2022)
- [Structural Restricted Boltzmann Machine for image denoising and classification](https://mlsanity.com/paper/2306.09628/) (2023)
- [Implicit Variational Rejection Sampling](https://mlsanity.com/paper/2606.14235/) (2026)

## Abstract (arXiv)

High-dimensional generative models have many applications including image compression, multimedia generation, anomaly detection and data completion. State-of-the-art estimators for natural images are autoregressive, decomposing the joint distribution over pixels into a product of conditionals parameterized by a deep neural network, e.g. a convolutional neural network such as the PixelCNN. However, PixelCNNs only model a single decomposition of the joint, and only a single generation order is efficient. For tasks such as image completion, these models are unable to use much of the observed context. To generate data in arbitrary orders, we introduce LMConv: a simple modification to the standard 2D convolution that allows arbitrary masks to be applied to the weights at each location in the image. Using LMConv, we learn an ensemble of distribution estimators that share parameters but differ in generation order, achieving improved performance on whole-image density estimation (2.89 bpd on unconditional CIFAR10), as well as globally coherent image completions. Our code is available at https://ajayjain.github.io/lmconv.