# Improving Semantic Coverage in AI Test & Evaluation through Generative Augmentation

arXiv: https://arxiv.org/abs/2610.13697 · HTML: https://mlsanity.com/paper/2610.13697/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [Effective Data Augmentation With Diffusion Models](https://mlsanity.com/paper/2302.07944/) (2023)
- [Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges](https://mlsanity.com/paper/2102.12219/) (2021)
- [Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning](https://mlsanity.com/paper/2308.06038/) (2023)
- [Benchmarking Detection Transfer Learning with Vision Transformers](https://mlsanity.com/paper/2111.11429/) (2021)
- [One-Step Image Translation with Text-to-Image Models](https://mlsanity.com/paper/2403.12036/) (2024)
- [Scaling Laws of Synthetic Images for Model Training ... for Now](https://mlsanity.com/paper/2312.04567/) (2023)
- [Prompt Tuning Inversion for Text-Driven Image Editing Using Diffusion Models](https://mlsanity.com/paper/2305.04441/) (2023)
- [Expanding Small-Scale Datasets with Guided Imagination](https://mlsanity.com/paper/2211.13976/) (2022)
- [Vision-Language Models can Identify Distracted Driver Behavior from Naturalistic Videos](https://mlsanity.com/paper/2306.10159/) (2023)
- [Stable Diffusion For Aerial Object Detection](https://mlsanity.com/paper/2311.12345/) (2023)
- [Filtering Empty Camera Trap Images in Embedded Systems](https://mlsanity.com/paper/2104.08859/) (2021)
- [Object detection in adverse weather conditions for autonomous vehicles using Instruct Pix2Pix](https://mlsanity.com/paper/2505.08228/) (2025)
- [Synthetic Data Augmentation using Pre-trained Diffusion Models for Long-tailed Food Image Classification](https://mlsanity.com/paper/2506.01368/) (2025)
- [Diverse Generation while Maintaining Semantic Coordination: A Diffusion-Based Data Augmentation Method for Object Detection](https://mlsanity.com/paper/2408.02891/) (2024)

## Abstract (arXiv)

In test and evaluation (T&E) of AI models, evaluators typically have access only to a held-out test dataset and no knowledge of the training data or its semantic composition. If critical input conditions---such as time-of-day, weather, or scene location---are underrepresented in this test dataset, T&E results can be misleadingly due to this semantic coverage gap. We propose a three-stage generative AI pipeline to detect and fill such semantic gaps without requiring ground-truth labels of these semantics. First, vision-language models (VLMs) perform zero-shot binary labeling of semantic factors, and an embedding-space clustering step identifies which semantics most affect downstream model behavior. Second, diffusion-based image-to-image models transform existing covered images into the underrepresented semantic domain, preserving scene content while augmenting high-level appearance. Third, statistical analysis of the augmented image embeddings verifies that the synthetic imagery improves dataset coverage before it is incorporated. Crucially, the synthetic data is used only to enrich the test set and no model retraining is performed for T&E. We validate the full pipeline on both image classification and object detection under day/night semantic gaps, showing that the augmented test datasets better distinguish models with and without exposure to the missing semantic, thereby yielding T&E conclusions that more closely match those of an ideal, fully covered dataset. Code available at https://github.com/Hashe037/DEDUCE