Sketch2MinSurf: Vision-Language Guided Generation of Editable Minimal Surfaces from Hand-Drawn Sketches
Not on the map. Why?
Abstract (arXiv)
Converting hand-drawn sketches into structured 3D geometries remains challenging due to the difficulty of representing non-Euclidean surfaces and maintaining topological consistency. Existing generative models such as GANs, NeRFs, and diffusion architectures often fail to produce editable manifolds directly usable in downstream design workflows. We present Sketch2MinSurf, a hybrid vision-language and geometric optimization framework that integrates vision-language guidance with minimal-surface theory to generate smooth and editable 3D surfaces from hand-drawn sketches. The core of our approach is a spatial-topological encoding that represents geometry as tuples of node coordinates and real/virtual edge skeletons, enabling stable topological control during generation. We further introduce the Sketch2MinSurf Structural Loss (S2MS-Loss), a reward-modulated objective that jointly constrains geometric reconstruction and topological coherence. On a test set of 100 sketches, Sketch2MinSurf achieves a topological similarity score of 0.844, outperforming existing sketch-to-shape baselines. The generated manifolds are directly editable and free from non-manifold artifacts. A public art installation at a university showcases the method's potential for human-intent-driven 3D form generation. The dataset and code are available at https://anonymous.4open.science/r/Sketch2MinSurf/.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
29 of 44 references.
References · 29
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Segment Anythingcited by 8,158
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Generative Adversarial Networkscited by 5,603
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
17 more
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
- 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
- 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
- 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
- 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
- 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusioncited by 204
- 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
- 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 132
- 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
- 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 94
- 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planningcited by 62
- 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priorscited by 53
- 2025MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaversecited by 50
- 2023BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Modelscited by 6
Built on by
Not cited yet.