# The Image Local Autoregressive Transformer

arXiv: https://arxiv.org/abs/2106.02514 · HTML: https://mlsanity.com/paper/2106.02514/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 57 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Large Scale Image Completion via Co-Modulated Generative Adversarial Networks](https://mlsanity.com/paper/2103.10428/) (2021)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [PixelSNAIL: An Improved Autoregressive Generative Model](https://mlsanity.com/paper/1712.09763/) (2017)
- [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020)

## Built on by (most cited first, 14 of 14)

- [N"UWA-LIP: Language Guided Image Inpainting with Defect-free VQGAN](https://mlsanity.com/paper/2202.05009/) (2022)
- [Exploring Efficient Few-shot Adaptation for Vision Transformers](https://mlsanity.com/paper/2301.02419/) (2023)
- [S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attention](https://mlsanity.com/paper/2210.12381/) (2022)
- [Human MotionFormer: Transferring Human Motions with Vision Transformers](https://mlsanity.com/paper/2302.11306/) (2023)
- [EdiBERT, a generative model for image editing](https://mlsanity.com/paper/2111.15264/) (2021)
- [ELMformer: Efficient Raw Image Restoration with a Locally Multiplicative Transformer](https://mlsanity.com/paper/2208.14704/) (2022)
- [Krause Synchronization Transformers](https://mlsanity.com/paper/2602.11534/) (2026)
- [ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions](https://mlsanity.com/paper/2205.12231/) (2022)
- [ViR: Towards Efficient Vision Retention Backbones](https://mlsanity.com/paper/2310.19731/) (2023)
- [What If : Understanding Motion Through Sparse Interactions](https://mlsanity.com/paper/2510.12777/) (2025)
- [Shaken, and Stirred: Long-Range Dependencies Enable Robust Outlier Detection with PixelCNN++](https://mlsanity.com/paper/2208.13579/) (2022)
- [QS-Craft: Learning to Quantize, Scrabble and Craft for Conditional Human Motion Animation](https://mlsanity.com/paper/2203.11632/) (2022)
- [Exploring Image Generation via Mutually Exclusive Probability Spaces and Local Correlation Hypothesis](https://mlsanity.com/paper/2506.21731/) (2025)
- [DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2606.01323/) (2026)

## Abstract (arXiv)

Recently, AutoRegressive (AR) models for the whole image generation empowered by transformers have achieved comparable or even better performance to Generative Adversarial Networks (GANs). Unfortunately, directly applying such AR models to edit/change local image regions, may suffer from the problems of missing global information, slow inference speed, and information leakage of local guidance. To address these limitations, we propose a novel model -- image Local Autoregressive Transformer (iLAT), to better facilitate the locally guided image synthesis. Our iLAT learns the novel local discrete representations, by the newly proposed local autoregressive (LA) transformer of the attention mask and convolution mechanism. Thus iLAT can efficiently synthesize the local image regions by key guidance information. Our iLAT is evaluated on various locally guided image syntheses, such as pose-guided person image synthesis and face editing. Both the quantitative and qualitative results show the efficacy of our model.