mlsanityData to 7 Oct 2026

GReFEM: Multimodal LLMs as Zero-Shot Semantic Assistants for Physics-Guided 3D Mesh Refinement

arXiv 2607.08798 ↗First posted 8 Jul 2026cs.GR cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Adaptive volumetric finite element meshing is a critical step in computer-aided engineering and analysis that dictates the computational budget of a given problem. It traditionally requires iterative PDE solvers or heavily supervised, data-driven surrogates trained on large-scale simulation data. While Multimodal Large Language Models (MLLMs) excel in 2D visual tasks, their zero-shot capability to semantically ground regions based on geometric understanding and physics remains an open question. Overall, this study explores a significant question: can the high-level semantic understanding of off-the-shelf MLLMs serve as a viable, zero-shot geometric proxy for finite element mesh refinement? To investigate this, we introduce GReFEM (Geometric Reasoning Enhanced Multimodal LLMs for Finite Element Meshing), a framework that utilizes MLLMs to visually localize stress-critical regions based on physics-guided textual prompts. To bridge the gap between 2D MLLM pre-training and 3D geometries, we introduce orthoViews, a view-selection module that maximizes the observability of key geometric features. We conduct an in-depth empirical evaluation across diverse CAD geometries, loading cases, and SOTA MLLMs, comparing them against a tuned geometric heuristic under a strict, matched refinement budget. Our findings reveal that MLLMs demonstrate robust zero-shot capacity to accurately follow complex spatial-physical instructions, isolating stress-relevant features with higher precision than blind heuristics. By mapping both the successes and current limitations of MLLMs in physical grounding, this study defines the frontier of foundation models as semantic assistants in automated simulation workflows.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 30 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

17 of 30 references.

Closest ancestors · 5

Same direction, newest first

  1. 20253D Question Answering via only 2D Vision-Language Models28 May
  2. 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding30 Nov
  3. 2024Language-Image Models with 3D Understanding6 May
  4. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors18 Mar
  5. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 12

  1. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  2. 2023LERF: Language Embedded Radiance Fieldscited by 538
  3. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 339
  4. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  5. 2023SAM3D: Segment Anything in 3D Scenescited by 155
  6. 2024LLaMA-Mesh: Unifying 3D Mesh Generation with Language Modelscited by 87
  7. 2024Generating CAD Code with Vision-Language Models for 3D Designscited by 61
  8. 2025MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Meshcited by 18
  9. 2024LLMI3D: MLLM-based 3D Perception from a Single 2D Imagecited by 7
  10. 2024QueryCAD: Grounded Question Answering for CAD Modelscited by 5
  11. 2025AMBER: Adaptive Mesh Generation by Iterative Mesh Resolution Predictioncited by 4
  12. 2024G-Adaptivity: optimised graph-based mesh relocation for finite element methodscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024LLMPhy: Parameter-Identifiable Physical Reasoning Combining Large Language Models and Physics Engines
  2. 2026EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers
  3. 2026Scaling-Aware Adapter for Structure-Grounded LLM Reasoning
  4. 2024Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions
  5. 2026VFEAgent: A Multimodal Agent Framework for End-to-End Automated Finite Element Analysis
  6. 2026PolyBridgeBench: Benchmarking Multimodal LLMs for Physics-Grounded Bridge Design
  7. 2026Concise Geometric Description as a Bridge: Unleashing the Potential of LLM for Plane Geometry Problem Solving
  8. 2026P3D-Bench: Benchmarking MLLMs for Parametric 3D Generation and Structural Reasoning