# FlowBERT: Prompt-tuned BERT for variable flow field prediction

arXiv: https://arxiv.org/abs/2506.08021 · HTML: https://mlsanity.com/paper/2506.08021/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition](https://mlsanity.com/paper/1710.04340/) (2017)
- [Neural Implicit Flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data](https://mlsanity.com/paper/2204.03216/) (2022)
- [Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flows](https://mlsanity.com/paper/2007.00936/) (2020)

## Built on by (most cited first, 1 of 1)

- [LLM4Fluid: Large Language Models as Generalizable Neural Solvers for Fluid Dynamics](https://mlsanity.com/paper/2601.21681/) (2026)

## Abstract (arXiv)

This study proposes a universal flow field prediction framework based on knowledge transfer from large language model (LLM), addressing the high computational costs of traditional computational fluid dynamics (CFD) methods and the limited cross-condition transfer capability of existing deep learning models. The framework innovatively integrates Proper Orthogonal Decomposition (POD) dimensionality reduction with fine-tuning strategies for pretrained LLM, where POD facilitates compressed representation of flow field features while the fine-tuned model learns to encode system dynamics in state space. To enhance the model's adaptability to flow field data, we specifically designed fluid dynamics-oriented text templates that improve predictive performance through enriched contextual semantic information. Experimental results demonstrate that our framework outperforms conventional Transformer models in few-shot learning scenarios while exhibiting exceptional generalization across various inflow conditions and airfoil geometries. Ablation studies reveal the contributions of key components in the FlowBERT architecture. Compared to traditional Navier-Stokes equation solvers requiring hours of computation, our approach reduces prediction time to seconds while maintaining over 90% accuracy. The developed knowledge transfer paradigm establishes a new direction for rapid fluid dynamics prediction, with potential applications extending to aerodynamic optimization, flow control, and other engineering domains.