# Mesh-Native Tokenisation: Retrofitting Foundation Models to Unstructured Data

arXiv: https://arxiv.org/abs/2610.12618 · HTML: https://mlsanity.com/paper/2610.12618/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 31 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 27 of 27)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://mlsanity.com/paper/2402.02366/) (2024)
- [Geometry-Informed Neural Operator for Large-Scale 3D PDEs](https://mlsanity.com/paper/2309.00583/) (2023)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels](https://mlsanity.com/paper/1711.08920/) (2017)
- [The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning](https://mlsanity.com/paper/2412.00568/) (2024)
- [Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries](https://mlsanity.com/paper/2502.02414/) (2025)
- [Operator Learning with Neural Fields: Tackling PDEs on General Geometries](https://mlsanity.com/paper/2306.07266/) (2023)
- [Latent Neural Operator for Solving Forward and Inverse PDE Problems](https://mlsanity.com/paper/2406.03923/) (2024)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [Walrus: A Cross-Domain Foundation Model for Continuum Dynamics](https://mlsanity.com/paper/2511.15684/) (2025)
- [CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs](https://mlsanity.com/paper/2505.12944/) (2025)
- [Probabilistic Retrofitting of Learned Simulators](https://mlsanity.com/paper/2603.01949/) (2026)
- [The Seismic Wavefield Common Task Framework](https://mlsanity.com/paper/2512.19927/) (2025)
- [Overtone: Cyclic Patch Modulation for Clean, Efficient, and Flexible Physics Emulators](https://mlsanity.com/paper/2507.09264/) (2025)
- [CTF4Nuclear: Common Task Framework for Nuclear Fission and Fusion Models](https://mlsanity.com/paper/2605.15549/) (2026)
- [A fast neural hybrid Newton solver adapted to implicit methods for nonlinear dynamics](https://mlsanity.com/paper/2407.03945/) (2024)
- [G-Adaptivity: optimised graph-based mesh relocation for finite element methods](https://mlsanity.com/paper/2407.04516/) (2024)
- [Discretization Invariant Networks for Learning Maps between Neural Fields](https://mlsanity.com/paper/2206.01178/) (2022)
- [Physics-informed reduced-order modelling with equivariant spectral submanifolds](https://mlsanity.com/paper/2608.04239/) (2026)

## Abstract (arXiv)

Physics foundation models are an increasingly popular approach for building versatile emulators for spatiotemporal systems. They require tokenisation layers that lift physical field data into a latent representation suitable for dynamic processing. A typical choice for this layer is a convolutional encoder, which lifts field information in a locality-aware way and can be trained very efficiently. However, this choice cannot be applied to a large class of scientific computing problems where data are represented on unstructured meshes, for example simulations on domains with curved boundaries. Such data typically require graph neural network architectures, which are more expensive to train and, in the form currently used, encode information differently from their CNN counterparts, thus limiting the portability of physics emulators between these data regimes. In this work we introduce Mesh-Native Convolution (MNC), a locality-aware tokenisation layer that encodes unstructured mesh-based data in a way that is analogous to CNNs. This approach leads to models that can be trained once on mesh data and then deployed to arbitrary unstructured meshes, even if these are irregular, with minimal retraining. We show that this tokenisation approach can be used to retrofit grid-pretrained foundation models efficiently on unstructured meshes and provide theoretical results to confirm the consistency of this approach with classical CNN encoders. We demonstrate the favourable performance of this retrofitting approach, which in many cases reaches acceptable accuracy even in zero-shot deployment, and outperforms existing baselines after only a few epochs of fine-tuning, thus providing an efficient bridge between the two data regimes.