ControlTrace: Recovering Control Fields for Hidden-Content Recognition
On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment also fits: Prompt learning and adaptation of CLIP
also in 1 more
Abstract (arXiv)
Spatially conditioned diffusion models can embed words and contours in natural-looking images, but vision-language models (VLMs) may fail to recognize the hidden content. Transformation-based recovery depends on parameter and view selection. To evaluate hidden-content recovery and recognition, we construct FreqBlind, a 6,000-image benchmark spanning contours, real words and non-words across three conditioning strengths. The evaluated transformation-based methods show limited recognition of contour patterns and weakly conditioned hidden content. To address this limitation, we propose ControlTrace to recover the grayscale control field used during generation. An 8.4M-parameter U-Net predicts this field from the carrier image, and a VLM then identifies its content. With Qwen2.5-VL-7B-Instruct, ControlTrace achieves 60.2% open-ended contour recognition accuracy across the three conditioning strengths, exceeding the best of the three evaluated prior methods by 26.9 percentage points. On an A100 GPU, the complete pipeline adds only 7.4 ms (5.3%) to direct VLM inference. Recovered fields have lower pixel errors and higher structural similarity than the evaluated transformation views. Across four evaluated VLMs, ControlTrace retains its overall contour recognition advantage. Recognition remains stable under the tested JPEG compression, Gaussian noise and downsampling. These results support control-field recovery for hidden-content recognition in the evaluated setting.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 20 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
15 of 20 references.
References · 15
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2022Simple Baselines for Image Restorationcited by 708
- 2019FIGR: Few-shot Image Generation with Reptilecited by 56
- 2025SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinkingcited by 6
- 2025Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusionscited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Heeding the Inner Voice: Aligning ControlNet Training via Intermediate Features Feedback
- 2025FreeControl: Efficient, Training-Free Structural Control via One-Step Attention Extraction
- 2026Improving Controllable Generation: Faster Training and Better Performance via x0-Supervision
- 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance
- 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models
- 2026PixelControl: Fine-Grained Condition Fidelity in Text-to-Image Diffusion
- 2025Att-Adapter: A Robust and Precise Domain-Specific Multi-Attributes T2I Diffusion Adapter via Conditional Variational Autoencoder
- 2026Language-Free Generative Editing from One Visual Example