# Text-to-Image Generation for Projector-Camera System Registration

arXiv: https://arxiv.org/abs/2607.03046 · HTML: https://mlsanity.com/paper/2607.03046/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 3 Jul 2026.

## Builds on (most cited first, 10 of 10)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color](https://mlsanity.com/paper/2109.06129/) (2021)
- [Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation](https://mlsanity.com/paper/2303.09319/) (2023)
- [Image-to-Graph Convolutional Network for Deformable Shape Reconstruction from a Single Projection Image](https://mlsanity.com/paper/2108.12533/) (2021)
- [CompenNet++: End-to-end Full Projector Compensation](https://mlsanity.com/paper/1908.06246/) (2019)
- [SGE: Structured Light System Based on Gray Code with an Event Camera](https://mlsanity.com/paper/2403.07326/) (2024)
- [Diffusion-based Aesthetic QR Code Generation via Scanning-Robust Perceptual Guidance](https://mlsanity.com/paper/2403.15878/) (2024)

## Abstract (arXiv)

Establishing correspondence between projector and camera images in a procam (projector + camera) system is essential for achieving high-resolution pixel matching, referred to as procam registration. The highest accuracy is typically obtained using structured light patterns (e.g., stripes or blobs). However, these methods are often inefficient and lack meaningful information for human viewers. Although some have explored the use of natural images, these often fail to provide a sufficient distribution of features to achieve comparable accuracy. Additionally, existing methods struggle to cope with environmental factors such as surface textures and variations in brightness due to ambient light or changes in camera exposure. To address these limitations, we propose a method based on deep neural networks. Our approach aims to generate a single natural image from text-based prompts that not only appears realistic but also possesses rich spatial features to enhance registration accuracy in procam applications. We have developed a deep neural network trained on a synthesized dataset that simulates potential geometric and photometric distortions encountered in a procam system illuminating a relatively smooth object (see Figure 1). Our trained network predicts the correspondence between projector and camera images, significantly improving registration accuracy across various procam configurations. By jointly considering the naturalness and feature richness of the projector images, our method minimizes visual disruptions in projected content without sacrificing precision. A user study confirms that our technique enhances perceived naturalness and usability compared to existing methods, validating its practical utility in real-world applications.