Distance-Aware Attention and Wall-Distance Expert Routing for Transformer-Based 3D Flow Prediction
On the map: Mixture-of-experts → Vision mixture-of-experts
Abstract (arXiv)
Transformer surrogates for 3D flow prediction compress an industrial mesh into a small set of tokens from which every prediction point reads. Two operations follow: the retrieval step in which a point gathers information from the compressed representation, and the feed-forward layer that transforms what it retrieved. In current backbones both are blind to where the point sits in the flow. We condition both on wall-related physical signals. Distance-aware cross-attention (DA-CA) reshapes each volume query by its wall distance before retrieval, so that a point deep in the boundary layer draws different geometric information than one in the outer flow. Surface-volume mixture-of-experts (SVMoE) replaces the shared feed-forward layer with a small set of experts, routed by wall distance for volume points and by local geometry for surface points. Neither mechanism is tied to one architecture, so we apply both unchanged to AB-UPT and Transolver-3. On DrivAerML with 50 training cases, DA-CA reduces the volume pressure error by 10.1%, and DA-CA and SVMoE together reduce it by 12.5%; DA-CA improves the near-wall region at some cost in the far region, which SVMoE recovers, and the volume experts settle into near-wall, transition, and free-stream bands without routing supervision. Retrained on 300 cases, the conditioning improves every field quantity, reducing volume pressure and velocity errors by 33.1% and 18.6% on AB-UPT and by 21.4% and 21.3% on Transolver-3. Under Leave-One-Body-Out evaluation on DrivAerNet++, it reduces the volume pressure error on unseen body types by up to 14.2%.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 24 of its 30 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
24 of 30 references.
Closest ancestors · 1
Same direction, newest first
Other references · 23
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
- 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 326
- 2023An Expert's Guide to Training Physics-informed Neural Networkscited by 126
- 2025Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometriescited by 97
- 2024DrivAerNet++: A Large-Scale Multimodal Car Dataset with Computational Fluid Dynamics Simulations and Deep Learning Benchmarkscited by 75
- 2025Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domainscited by 62
- 2025AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformerscited by 53
11 more
- 2024Geom-DeepONet: A Point-cloud-based Deep Operator Network for Field Predictions on 3D Parameterized Geometriescited by 47
- 2024DrivAerML: High-Fidelity Computational Fluid Dynamics Dataset for Road-Car External Aerodynamicscited by 35
- 2025DoMINO: A Decomposable Multi-scale Iterative Neural Operator for Modeling Large Scale Engineering Simulationscited by 27
- 2025GeoTransolver: Learning Physics on Irregular Domains Using Multi-scale Geometry Aware Physics Attention Transformercited by 23
- 2025Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systemscited by 22
- 2026Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometriescited by 20
- 2024Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspectivecited by 12
- 2025Towards scalable surrogate models based on Neural Fields for large scale aerodynamic simulationscited by 10
- 2025Data-Efficient Deep Operator Network for Unsteady Flow: A Multi-Fidelity Approach with Physics-Guided Subsamplingcited by 5
- 2025A Mixture of Experts Gating Network for Enhanced Surrogate Modeling in External Aerodynamicscited by 2
- 2026GA-Field: Geometry-Aware Vehicle Aerodynamic Field Predictioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MeshFlow: Mesh Generation with Equivariant Flow Matching
- 2025A Study of Finetuning Video Transformers for Multi-view Geometry Tasks
- 2026Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding
- 2026FluidFlow: a flow-matching generative model for fluid dynamics surrogates on unstructured meshes
- 2026Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow
- 20253DID: Direct 3D Inverse Design for Aerodynamics with Physics-Aware Optimization
- 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attention
- 2026Attention Itself Could Retrieve.RetrieveVGGT: Training-Free Long Context Streaming 3D Reconstruction via Query-Key Similarity Retrieval