# LAPIG: Language Guided Projector Image Generation with Surface Adaptation and Stylization

arXiv: https://arxiv.org/abs/2503.12173 · HTML: https://mlsanity.com/paper/2503.12173/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 61 references that are in the map.
- First posted on arXiv 15 Mar 2025.

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [SpaText: Spatio-Textual Representation for Controllable Image Generation](https://mlsanity.com/paper/2211.14305/) (2022)
- [GALIP: Generative Adversarial CLIPs for Text-to-Image Synthesis](https://mlsanity.com/paper/2301.12959/) (2023)
- [SPAA: Stealthy Projector-based Adversarial Attacks on Deep Image Classifiers](https://mlsanity.com/paper/2012.05858/) (2020)
- [End-to-end Full Projector Compensation](https://mlsanity.com/paper/2008.00965/) (2020)
- [End-to-end Projector Photometric Compensation](https://mlsanity.com/paper/1904.04335/) (2019)
- [Neural Projection Mapping Using Reflectance Fields](https://mlsanity.com/paper/2306.06595/) (2023)
- [Casper DPM: Cascaded Perceptual Dynamic Projection Mapping onto Hands](https://mlsanity.com/paper/2409.04397/) (2024)
- [CompenHR: Efficient Full Compensation for High-resolution Projector](https://mlsanity.com/paper/2311.13409/) (2023)
- [Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop Alignment](https://mlsanity.com/paper/1909.00032/) (2019)
- [Stay Positive: Non-Negative Image Synthesis for Augmented Reality](https://mlsanity.com/paper/2202.00659/) (2022)

## Built on by (most cited first, 2 of 2)

- [ProCap: Projection-Aware Captioning for Spatial Augmented Reality](https://mlsanity.com/paper/2604.00912/) (2026)
- [Physically-guided Image Generation for Multi-Projection Mapping](https://mlsanity.com/paper/2606.22477/) (2026)

## Abstract (arXiv)

We propose LAPIG, a language guided projector image generation method with surface adaptation and stylization. LAPIG consists of a projector-camera system and a target textured projection surface. LAPIG takes the user text prompt as input and aims to transform the surface style using the projector. LAPIG's key challenge is that due to the projector's physical brightness limitation and the surface texture, the viewer's perceived projection may suffer from color saturation and artifacts in both dark and bright regions, such that even with the state-of-the-art projector compensation techniques, the viewer may see clear surface texture-related artifacts. Therefore, how to generate a projector image that follows the user's instruction while also displaying minimum surface artifacts is an open problem. To address this issue, we propose projection surface adaptation (PSA) that can generate compensable surface stylization. We first train two networks to simulate the projector compensation and project-and-capture processes, this allows us to find a satisfactory projector image without real project-and-capture and utilize gradient descent for fast convergence. Then, we design content and saturation losses to guide the projector image generation, such that the generated image shows no clearly perceivable artifacts when projected. Finally, the generated image is projected for visually pleasing surface style morphing effects. The source code and video are available on the project page: https://Yu-chen-Deng.github.io/LAPIG/.