# GReFEM: Multimodal LLMs as Zero-Shot Semantic Assistants for Physics-Guided 3D Mesh Refinement

arXiv: https://arxiv.org/abs/2607.08798 · HTML: https://mlsanity.com/paper/2607.08798/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 17 of 17)

- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [PointLLM: Empowering Large Language Models to Understand Point Clouds](https://mlsanity.com/paper/2308.16911/) (2023)
- [SAM3D: Segment Anything in 3D Scenes](https://mlsanity.com/paper/2306.03908/) (2023)
- [Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding](https://mlsanity.com/paper/2412.00493/) (2024)
- [LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models](https://mlsanity.com/paper/2411.09595/) (2024)
- [Generating CAD Code with Vision-Language Models for 3D Designs](https://mlsanity.com/paper/2410.05340/) (2024)
- [SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors](https://mlsanity.com/paper/2403.13438/) (2024)
- [Language-Image Models with 3D Understanding](https://mlsanity.com/paper/2405.03685/) (2024)
- [MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Mesh](https://mlsanity.com/paper/2508.01242/) (2025)
- [3D Question Answering via only 2D Vision-Language Models](https://mlsanity.com/paper/2505.22143/) (2025)
- [LLMI3D: MLLM-based 3D Perception from a Single 2D Image](https://mlsanity.com/paper/2408.07422/) (2024)
- [QueryCAD: Grounded Question Answering for CAD Models](https://mlsanity.com/paper/2409.08704/) (2024)
- [AMBER: Adaptive Mesh Generation by Iterative Mesh Resolution Prediction](https://mlsanity.com/paper/2505.23663/) (2025)
- [G-Adaptivity: optimised graph-based mesh relocation for finite element methods](https://mlsanity.com/paper/2407.04516/) (2024)

## Abstract (arXiv)

Adaptive volumetric finite element meshing traditionally requires iterative PDE solvers or heavily supervised, data-driven surrogates trained on large-scale simulation data. This work explores a significant question: can the high-level semantic understanding of off-the-shelf MLLMs serve as a viable, zero-shot geometric proxy for finite element mesh refinement? We introduce GReFEM (Geometric Reasoning Enhanced Multimodal LLMs for Finite Element Meshing), a framework that utilizes MLLMs to visually localize stress-critical regions based on physics-guided textual prompts. To bridge the gap between 2D MLLM pre-training and 3D geometries, we introduce textitorthoViews, a trained view-selection module that assists these MLLMs by maximizing the observability of key geometric features. We evaluate GReFEM by computing finite-element error at stress concentrations, on diverse CAD mesh geometries under multiple loading cases, with MLLMs and prompt variants, against blind and load-informed geometric heuristics, solver-informed Zienkiewicz--Zhu (ZZ) refinement, and a retrained learned adaptive-meshing baseline, all at a matched element budget. Without any FEM solve or simulation training data, GReFEM lowers the critical-region peak von Mises stress error relative to a blind geometric heuristic and to refinement driven by a coarse FEM solve, closing 58% of the gap between the unrefined mesh and a reference-solve oracle, while a hand-derived analytical load-informed heuristic remains stronger and the learned baseline performs slightly better overall. With this we investigate their performance as semantic assistants in automated simulation workflows. Code is available at https://github.com/kbali1297/GReFEM