mlsanityData to 14 Oct 2026

MV-GEL: Language-Driven Multi-View Geometric Entity Localization on Meshes

arXiv 2606.31533 ↗First posted 30 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Identifying and grounding precise geometric entities, such as edges, planar regions, and curved surfaces within 3D objects, is foundational to computer-aided design (CAD), robotic manipulation, and scientific simulation. Although modern Vision Language Models (VLMs) have advanced referring segmentation (RIS) in the image domain, extending such language-driven localization to structured 3D geometry is substantially harder. The 3D object appearance is highly sensitive to viewpoints; a single perspective may render a target entity clearly observable, while another may suffer from severe occlusion or foreshortening. In this work, we attempt to solve these challenges with MV-GEL (Multi-View Geometric Entity Localization), a framework for localizing fine-grained geometric entities on polygon meshes from natural language queries. Our key insight is that reliable CAD entity (i.e., faces, edges or solids) localization depends on selecting views that make the queried entity maximally interpretable. We introduce GELviews, a prompt-conditioned ranking module that prioritizes viewpoints based on language prompted observability of geometric CAD entities. Selected views are processed by a VLM-based reasoning segmentation backbone, and predicted masks are lifted to the corresponding meshes via geometry-aware ray casting. Our framework is completely CAD agnostic and relies only on 3D meshes. Experiments show up to a 1.7X improvement in face-level IoU and over 4.5X gains in edge-level F1 compared to vanilla baselines, substantially outperforming CLIP-based and random view sampling, particularly for thin and view-sensitive structures. The dataset, code and trained checkpoints are available at https://github.com/kbali1297/MV-GEL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 50 of its 56 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

50 of 56 references.

References · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023Segment Anythingcited by 8,234
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
38 more
  1. 2021Grounded Language-Image Pre-trainingcited by 1,156
  2. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  3. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 565
  4. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  5. 2023Segment Everything Everywhere All at Oncecited by 548
  6. 2023LERF: Language Embedded Radiance Fieldscited by 543
  7. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  8. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  9. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  10. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 341
  11. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  12. 2023Fast Segment Anythingcited by 290
  13. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  14. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  15. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 222
  16. 2023Scalable 3D Captioning with Pretrained Modelscited by 203
  17. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 194
  18. 2022PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learningcited by 187
  19. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 159
  20. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 154
  21. 2021Revisiting Point Cloud Shape Classification with a Simple and Effective Baselinecited by 145
  22. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 118
  23. 2021BRepNet: A topological message passing system for solid modelscited by 95
  24. 2024CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLMcited by 81
  25. 2022SolidGen: An Autoregressive Model for Direct B-rep Synthesiscited by 80
  26. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  27. 2025CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generationcited by 67
  28. 2023SATR: Zero-Shot Semantic Segmentation of 3D Shapescited by 62
  29. 2024GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstructioncited by 39
  30. 2023PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimationcited by 37
  31. 2024Find Any Part in 3Dcited by 32
  32. 2019Dominant Set Clustering and Pooling for Multi-View 3D Object Recognitioncited by 25
  33. 2025ReferSplat: Referring Segmentation in 3D Gaussian Splattingcited by 20
  34. 20243D Part Segmentation via Geometric Aggregation of 2D Visual Featurescited by 15
  35. 2023Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understandingcited by 13
  36. 2024PARIS3D: Reasoning-based 3D Part Segmentation Using Large Multimodal Modelcited by 9
  37. 2025IPDN: Image-enhanced Prompt Decoding Network for 3D Referring Expression Segmentationcited by 8
  38. 2026PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understandingcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Toward an Artificial General Teacher: Procedural Geometry Data Generation and Visual Grounding with Vision-Language Models
  2. 2026Vision2CAD: A Visual Agent Harness for Explicit Geometry Referencing and Localization in Parametric CAD Modeling
  3. 2026Memory Over Maps: 3D Object Localization Without Reconstruction
  4. 2024Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models
  5. 2026StrataVLA: Hierarchical and Efficient 3D Geometric Grounding for Vision-Language-Action Models
  6. 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
  7. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
  8. 2026PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models