# AirFormer: Predicting Nationwide Air Quality in China with Transformers

arXiv: https://arxiv.org/abs/2211.15979 · HTML: https://mlsanity.com/paper/2211.15979/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 29 Nov 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [GMAN: A Graph Multi-Attention Network for Traffic Prediction](https://mlsanity.com/paper/1911.08415/) (2019)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [Ladder Variational Autoencoders](https://mlsanity.com/paper/1602.02282/) (2016)
- [Rethinking and Improving Relative Position Encoding for Vision Transformer](https://mlsanity.com/paper/2107.14222/) (2021)
- [Spatial-Temporal Transformer Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2001.02908/) (2020)
- [Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2206.09113/) (2022)
- [PM2.5-GNN: A Domain Knowledge Enhanced Graph Neural Network For PM2.5 Forecasting](https://mlsanity.com/paper/2002.12898/) (2020)
- [Group-Aware Graph Neural Network for Nationwide City Air Quality Forecasting](https://mlsanity.com/paper/2108.12238/) (2021)
- [STCN: Stochastic Temporal Convolutional Networks](https://mlsanity.com/paper/1902.06568/) (2019)
- [HighAir: A Hierarchical Graph Neural Network-Based Air Quality Forecasting Method](https://mlsanity.com/paper/2101.04264/) (2021)
- [HyperST-Net: Hypernetworks for Spatio-Temporal Forecasting](https://mlsanity.com/paper/1809.10889/) (2018)

## Built on by (most cited first, 30 of 73)

- [Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey](https://mlsanity.com/paper/2303.14483/) (2023)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models](https://mlsanity.com/paper/2301.13629/) (2023)
- [CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting](https://mlsanity.com/paper/2305.12095/) (2023)
- [Deciphering Spatio-Temporal Graph Forecasting: A Causal Lens and Treatment](https://mlsanity.com/paper/2309.13378/) (2023)
- [Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook](https://mlsanity.com/paper/2402.19348/) (2024)
- [AirPhyNet: Harnessing Physics-Guided Neural Networks for Air Quality Prediction](https://mlsanity.com/paper/2402.03784/) (2024)
- [Air Quality Prediction with Physics-Guided Dual Neural ODEs in Open Systems](https://mlsanity.com/paper/2410.19892/) (2024)
- [STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Model](https://mlsanity.com/paper/2403.12418/) (2024)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [PromptST: Prompt-Enhanced Spatio-Temporal Multi-Attribute Prediction](https://mlsanity.com/paper/2309.09500/) (2023)
- [Graph Neural Processes for Spatio-Temporal Extrapolation](https://mlsanity.com/paper/2305.18719/) (2023)
- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [Spatio-Temporal Field Neural Networks for Air Quality Inference](https://mlsanity.com/paper/2403.02354/) (2024)
- [Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting](https://mlsanity.com/paper/2410.12593/) (2024)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [ST-LoRA: Low-rank Adaptation for Spatio-Temporal Forecasting](https://mlsanity.com/paper/2404.07919/) (2024)
- [GBT: Two-stage transformer framework for non-stationary time series forecasting](https://mlsanity.com/paper/2307.08302/) (2023)
- [Machine Learning for Urban Air Quality Analytics: A Survey](https://mlsanity.com/paper/2310.09620/) (2023)
- [ARROW: An Adaptive Rollout and Routing Method for Global Weather Forecasting](https://mlsanity.com/paper/2510.09734/) (2025)
- [On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting](https://mlsanity.com/paper/2408.09695/) (2024)
- [Quantformer: from attention to profit with a quantitative transformer trading strategy](https://mlsanity.com/paper/2404.00424/) (2024)
- [Instructor-Worker Large Language Model System for Policy Recommendation: a Case Study on Air Quality Analysis of the January 2025 Los Angeles Wildfires](https://mlsanity.com/paper/2503.00566/) (2025)
- [NRFormer: Nationwide Nuclear Radiation Forecasting with Spatio-Temporal Transformer](https://mlsanity.com/paper/2410.11924/) (2024)
- [Spatiotemporal Causal Decoupling Model for Air Quality Forecasting](https://mlsanity.com/paper/2505.20119/) (2025)
- [OATS: Online Data Augmentation for Time Series Foundation Models](https://mlsanity.com/paper/2601.19040/) (2026)
- [Unleashing Realistic Air Quality Forecasting: Introducing the Ready-to-Use PurpleAirSF Dataset](https://mlsanity.com/paper/2306.13948/) (2023)
- [TPRNN: A Top-Down Pyramidal Recurrent Neural Network for Time Series Forecasting](https://mlsanity.com/paper/2312.06328/) (2023)
- [AirRadar: Inferring Nationwide Air Quality in China with Deep Neural Networks](https://mlsanity.com/paper/2501.13141/) (2025)
- [Urban Computing in the Era of Large Language Models](https://mlsanity.com/paper/2504.02009/) (2025)

## Abstract (arXiv)

Air pollution is a crucial issue affecting human health and livelihoods, as well as one of the barriers to economic and social growth. Forecasting air quality has become an increasingly important endeavor with significant social impacts, especially in emerging countries like China. In this paper, we present a novel Transformer architecture termed AirFormer to collectively predict nationwide air quality in China, with an unprecedented fine spatial granularity covering thousands of locations. AirFormer decouples the learning process into two stages -- 1) a bottom-up deterministic stage that contains two new types of self-attention mechanisms to efficiently learn spatio-temporal representations; 2) a top-down stochastic stage with latent variables to capture the intrinsic uncertainty of air quality data. We evaluate AirFormer with 4-year data from 1,085 stations in the Chinese Mainland. Compared to the state-of-the-art model, AirFormer reduces prediction errors by 5%~8% on 72-hour future predictions. Our source code is available at https://github.com/yoshall/airformer.