# Towards a Foundation-Model Paradigm for Aerodynamic Prediction in Three-dimensional Design

arXiv: https://arxiv.org/abs/2604.18062 · HTML: https://mlsanity.com/paper/2604.18062/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Towards a Unified View of Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2110.04366/) (2021)
- [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://mlsanity.com/paper/2402.02366/) (2024)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows](https://mlsanity.com/paper/1810.08217/) (2018)
- [ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks](https://mlsanity.com/paper/2408.11104/) (2024)
- [Machine Learning in Aerodynamic Shape Optimization](https://mlsanity.com/paper/2202.07141/) (2022)
- [Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers](https://mlsanity.com/paper/2405.17527/) (2024)
- [DrivAerML: High-Fidelity Computational Fluid Dynamics Dataset for Road-Car External Aerodynamics](https://mlsanity.com/paper/2408.11969/) (2024)
- [PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations](https://mlsanity.com/paper/2505.24717/) (2025)
- [Data-Driven Aerospace Engineering: Reframing the Industry with Machine Learning](https://mlsanity.com/paper/2008.10740/) (2020)
- [Enhanced data efficiency using deep neural networks and Gaussian processes for aerodynamic design optimization](https://mlsanity.com/paper/2008.06731/) (2020)
- [Parametric Generative Schemes with Geometric Constraints for Encoding and Synthesizing Airfoils](https://mlsanity.com/paper/2205.02458/) (2022)
- [Going with the Speed of Sound: Pushing Neural Surrogates into Highly-turbulent Transonic Regimes](https://mlsanity.com/paper/2511.21474/) (2025)
- [ONERA's CRM WBPN database for machine learning activities, related regression challenge and first results](https://mlsanity.com/paper/2505.06265/) (2025)
- [Rapid aerodynamic prediction of swept wings via physics-embedded transfer learning](https://mlsanity.com/paper/2409.12711/) (2024)
- [MMET: A Multi-Input and Multi-Scale Transformer for Efficient PDEs Solving](https://mlsanity.com/paper/2506.17230/) (2025)
- [Uncertainty-Aware Data-Based Method for Fast and Reliable Shape Optimization](https://mlsanity.com/paper/2601.21956/) (2026)

## Abstract (arXiv)

Accurate machine-learning models for aerodynamic prediction are essential for accelerating shape optimization, yet remain challenging to develop for complex three-dimensional configurations due to the high cost of generating training data. This work introduces a methodology for efficiently constructing accurate surrogate models for design purposes by first pre-training a large-scale model on diverse geometries and then fine-tuning it with a few more detailed task-specific samples. A Transformer-based architecture, AeroTransformer, is developed and tailored for large-scale training to learn aerodynamics. The methodology is evaluated on transonic wings, where the model is pre-trained on SuperWing, a dataset of nearly 30000 samples with broad geometric diversity, and subsequently fine-tuned to handle specific wing shapes perturbed from the Common Research Model. Results show that, with 450 task-specific samples, the proposed methodology achieves 0.36% error on surface-flow prediction, reducing 84.2% compared to training from scratch. The influence of model configurations and training strategies is also systematically studied to provide guidance on effectively training and deploying such models under limited data and computational budgets. To facilitate reuse, we release the datasets and the pre-trained models at https://github.com/tum-pbs/AeroTransformer. An interactive design tool is also built on the pre-trained model and is available online at https://webwing.pbs.cit.tum.de.