# NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis

arXiv: https://arxiv.org/abs/2207.09814 · HTML: https://mlsanity.com/paper/2207.09814/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 54 references that are in the map.
- First posted on arXiv 20 Jul 2022.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Compressive Transformers for Long-Range Sequence Modelling](https://mlsanity.com/paper/1911.05507/) (2019)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [GIT: A Generative Image-to-text Transformer for Vision and Language](https://mlsanity.com/paper/2205.14100/) (2022)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [Meshed-Memory Transformer for Image Captioning](https://mlsanity.com/paper/1912.08226/) (2019)
- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)

## Built on by (most cited first, 30 of 71)

- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [Phenaki: Variable Length Video Generation From Open Domain Textual Description](https://mlsanity.com/paper/2210.02399/) (2022)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation](https://mlsanity.com/paper/2303.12346/) (2023)
- [VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning](https://mlsanity.com/paper/2309.15091/) (2023)
- [Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising](https://mlsanity.com/paper/2305.18264/) (2023)
- [Vision-Language Pre-training: Basics, Recent Advances, and Future Trends](https://mlsanity.com/paper/2210.09263/) (2022)
- [CelebV-Text: A Large-Scale Facial Text-Video Dataset](https://mlsanity.com/paper/2303.14717/) (2023)
- [DiffCollage: Parallel Generation of Large Content with Diffusion Models](https://mlsanity.com/paper/2303.17076/) (2023)
- [DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation](https://mlsanity.com/paper/2412.18597/) (2024)
- [ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion](https://mlsanity.com/paper/2403.18818/) (2024)
- [InfiniCity: Infinite-Scale City Synthesis](https://mlsanity.com/paper/2301.09637/) (2023)
- [RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers](https://mlsanity.com/paper/2502.15894/) (2025)
- [VideoStudio: Generating Consistent-Content and Multi-Scene Videos](https://mlsanity.com/paper/2401.01256/) (2024)
- [RealFill: Reference-Driven Generation for Authentic Image Completion](https://mlsanity.com/paper/2309.16668/) (2023)
- [FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Models](https://mlsanity.com/paper/2406.16863/) (2024)
- [VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory](https://mlsanity.com/paper/2512.04519/) (2025)
- [MEVG: Multi-event Video Generation with Text-to-Video Models](https://mlsanity.com/paper/2312.04086/) (2023)
- [A Survey on Long Video Generation: Challenges, Methods, and Prospects](https://mlsanity.com/paper/2403.16407/) (2024)
- [Spatia: Video Generation with Updatable Spatial Memory](https://mlsanity.com/paper/2512.15716/) (2025)
- [Autoregressive Models in Vision: A Survey](https://mlsanity.com/paper/2411.05902/) (2024)
- [A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming](https://mlsanity.com/paper/2404.16038/) (2024)
- [Qwen-Image-Layered: Towards Inherent Editability via Layer Decomposition](https://mlsanity.com/paper/2512.15603/) (2025)
- [Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs](https://mlsanity.com/paper/2503.10084/) (2025)
- [World Models for Learning Dexterous Hand-Object Interactions from Human Videos](https://mlsanity.com/paper/2512.13644/) (2025)
- [StoryBench: A Multifaceted Benchmark for Continuous Story Visualization](https://mlsanity.com/paper/2308.11606/) (2023)
- [From Sora What We Can See: A Survey of Text-to-Video Generation](https://mlsanity.com/paper/2405.10674/) (2024)
- [LayerDiff: Exploring Text-guided Multi-layered Composable Image Synthesis via Layer-Collaborative Diffusion Model](https://mlsanity.com/paper/2403.11929/) (2024)
- [Playable Game Generation](https://mlsanity.com/paper/2412.00887/) (2024)

## Abstract (arXiv)

In this paper, we present NUWA-Infinity, a generative model for infinite visual synthesis, which is defined as the task of generating arbitrarily-sized high-resolution images or long-duration videos. An autoregressive over autoregressive generation mechanism is proposed to deal with this variable-size generation task, where a global patch-level autoregressive model considers the dependencies between patches, and a local token-level autoregressive model considers dependencies between visual tokens within each patch. A Nearby Context Pool (NCP) is introduced to cache-related patches already generated as the context for the current patch being generated, which can significantly save computation costs without sacrificing patch-level dependency modeling. An Arbitrary Direction Controller (ADC) is used to decide suitable generation orders for different visual synthesis tasks and learn order-aware positional embeddings. Compared to DALL-E, Imagen and Parti, NUWA-Infinity can generate high-resolution images with arbitrary sizes and support long-duration video generation additionally. Compared to NUWA, which also covers images and videos, NUWA-Infinity has superior visual synthesis capabilities in terms of resolution and variable-size generation. The GitHub link is https://github.com/microsoft/NUWA. The homepage link is https://nuwa-infinity.microsoft.com.