mlsanityData to 7 Oct 2026

GeoBiked: A Dataset with Geometric Features and Automated Labeling Techniques to Enable Deep Generative Models in Engineering Design

arXiv 2409.17045 ↗First posted 25 Sep 2024cs.CV cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

We provide a dataset for enabling Deep Generative Models (DGMs) in engineering design and propose methods to automate data labeling by utilizing large-scale foundation models. GeoBiked is curated to contain 4 355 bicycle images, annotated with structural and technical features and is used to investigate two automated labeling techniques: The utilization of consolidated latent features (Hyperfeatures) from image-generation models to detect geometric correspondences (e.g. the position of the wheel center) in structural images and the generation of diverse text descriptions for structural images. GPT-4o, a vision-language-model (VLM), is instructed to analyze images and produce diverse descriptions aligned with the system-prompt. By representing technical images as Diffusion-Hyperfeatures, drawing geometric correspondences between them is possible. The detection accuracy of geometric points in unseen samples is improved by presenting multiple annotated source images. GPT-4o has sufficient capabilities to generate accurate descriptions of technical images. Grounding the generation only on images leads to diverse descriptions but causes hallucinations, while grounding it on categorical labels restricts the diversity. Using both as input balances creativity and accuracy. Successfully using Hyperfeatures for geometric correspondence suggests that this approach can be used for general point-detection and annotation tasks in technical images. Labeling such images with text descriptions using VLMs is possible, but dependent on the models detection capabilities, careful prompt-engineering and the selection of input information. Applying foundation models in engineering design is largely unexplored. We aim to bridge this gap with a dataset to explore training, finetuning and conditioning DGMs in this field and suggesting approaches to bootstrap foundation models to process technical images.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 48 of its 64 references that are in the map. First posted on arXiv 25 Sep 2024.

Builds on

48 of 64 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2017Mask R-CNNcited by 7,259
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
36 more
  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2014Going Deeper with Convolutionscited by 4,231
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2023Mistral 7Bcited by 3,065
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  8. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  9. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  10. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  11. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  12. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  13. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  14. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  15. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  16. 2023Emergent Correspondence from Image Diffusioncited by 437
  17. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  18. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 211
  19. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  20. 2021Deep Generative Models in Engineering Design: A Reviewcited by 103
  21. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 100
  22. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Modelscited by 91
  23. 2022Zero-Shot Category-Level Object Pose Estimationcited by 57
  24. 2023DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedbackcited by 57
  25. 2024ImageInWords: Unlocking Hyper-Detailed Image Descriptionscited by 56
  26. 2023Readout Guidance: Learning Control from Diffusion Featurescited by 52
  27. 2023From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Designcited by 51
  28. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
  29. 2023Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Designcited by 22
  30. 2023Assessing GPT4-V on Structured Reasoning Taskscited by 14
  31. 2023On the Noise Scheduling for Generating Plausible Designs with Diffusion Modelscited by 6
  32. 2023Adversarial Latent Autoencoder with Self-Attention for Structural Image Synthesiscited by 5
  33. 2024InsertDiffusion: Identity Preserving Visualization of Objects through a Training-Free Diffusion Architecturecited by 5
  34. 2023Grounded Intuition of GPT-Vision's Abilities with Scientific Imagescited by 3
  35. 2024Fr'echet Denoised Distance: Enhancing Plausibility Evaluation for Generated Designs with Denoising Autoencodercited by 3
  36. 2024Exploring the Potentials and Challenges of Deep Generative Models in Product Design Conceptioncited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025MeshFleet: Filtered and Annotated 3D Vehicle Dataset for Domain Specific Generative Modelingcited by 1
  2. 2025Masked Conditioning for Deep Generative Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024CAD-Prompted Generative Models: A Pathway to Feasible and Novel Engineering Designs
  2. 2025From Drawings to Decisions: A Hybrid Vision-Language Framework for Parsing 2D Engineering Drawings into Structured Manufacturing Knowledge
  3. 2024CadVLM: Bridging Language and Vision in the Generation of Parametric CAD Sketches
  4. 2026Foundation Models for Automatic CAD Generation
  5. 2026SldprtNet: A Large-Scale Multimodal Dataset for CAD Generation in Language-Driven 3D Design
  6. 2024GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training
  7. 2025Deep Generative Methods and Tire Architecture Design
  8. 2026CADFS: A Big CAD Program Dataset and Framework for Computer-Aided Design with Large Language Models