Diffuse the object, keep its label: curating detector training data from a few unlabeled photographs via VLM-built 3D vegetation scenes
Not on the map. Why?
Abstract (arXiv)
Labeled images of small objects hidden in vegetation are scarce, and detectors trained on them generalize poorly across sites. Rather than reusing labels collected at another site, we synthesize labeled training images from a handful of unlabeled photographs of the deployment site itself. A vision--language model generates a coarse 3D vegetation scene from one photograph; placing 3D object meshes in the scene yields bounding boxes, segmentation masks, and per-instance occlusion directly from the scene geometry, without manual annotation. A lightweight adapter fine-tuned on the photographs conditions a diffusion pass that re-textures the renders, and a graded mask-lock sets how much diffusion may touch the object itself. In our runs this grade was the most influential curation choice: lightly diffusing the object improves minority-class recall over fully protecting its pixels, while unrestricted diffusion dissolves it. Trained on these images, a standard detector matched or exceeded its counterpart trained on a larger labeled dataset of real images from a different site, consistently across seeds on a humanitarian-demining benchmark; the comparison is thus unsupervised site adaptation from a handful of photographs against conventional cross-site label reuse. In our ablations the gains were largely insensitive to the photograph and crop budgets, and in-domain accuracy did not predict cross-site performance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
24 of 41 references.
References · 24
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2018Demystifying MMD GANscited by 1,293
- 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
- 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
- 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 294
- 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 134
12 more
- 2018Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Datacited by 94
- 2025R3D2: Realistic 3D Asset Insertion via Diffusion for Autonomous Driving Simulationcited by 18
- 2026Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Modelscited by 10
- 2026Coarse-to-Real: Generative Rendering for Populated Dynamic Scenescited by 9
- 2025MineInsight: A Multi-sensor Dataset for Humanitarian Demining Robotics in Off-Road Environmentscited by 5
- 2025Cut-and-Splat: Leveraging Gaussian Splatting for Synthetic Data Generationcited by 4
- 2024Leafy Spurge Dataset: Real-world Weed Classification Within Aerial Drone Imagerycited by 2
- 2024Rendering-Refined Stable Diffusion for Privacy Compliant Synthetic Datacited by 2
- 2025Exploring Model Quantization in GenAI-based Image Inpainting and Detection of Arable Plantscited by 2
- 2024A multi-robot system for the detection of explosive devicescited by 1
- 2025A Scalable Pipeline Combining Procedural 3D Graphics and Guided Diffusion for Photorealistic Synthetic Training Data Generation in White Button Mushroom Segmentationcited by 1
- 2026ForestPersons: A Large-Scale Dataset for Under-Canopy Missing Person Detectioncited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025FLORA: Efficient Synthetic Data Generation for Object Detection in Low-Data Regimes via finetuning Flux LoRA
- 2024SemCity: Semantic Scene Generation with Triplane Diffusion
- 2026Learning a Maximum Entropy Model for Visual Textures using Diffusion
- 2026SPROUT: A Scalable Diffusion Foundation Model for Multi-Crop Plant Phenotyping
- 2025Diverse Instance Generation via Diffusion Models for Enhanced Few-Shot Object Detection in Remote Sensing Images
- 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models
- 2026DiffuSAM: Diffusion Guided Zero-Shot Object Grounding for Remote Sensing Imagery
- 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Data