# RAVE: Residual Vector Embedding for CLIP-Guided Backlit Image Enhancement

arXiv: https://arxiv.org/abs/2404.01889 · HTML: https://mlsanity.com/paper/2404.01889/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement](https://mlsanity.com/paper/2001.06826/) (2020)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [DiffIR: Efficient Diffusion Model for Image Restoration](https://mlsanity.com/paper/2303.09472/) (2023)
- [Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement](https://mlsanity.com/paper/2303.06705/) (2023)
- [Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation](https://mlsanity.com/paper/2103.00860/) (2021)
- [Low-Light Image and Video Enhancement Using Deep Learning: A Survey](https://mlsanity.com/paper/2104.10729/) (2021)
- [Diff-Retinex: Rethinking Low-light Image Enhancement with A Generative Diffusion Model](https://mlsanity.com/paper/2308.13164/) (2023)
- [Iterative Prompt Learning for Unsupervised Backlit Image Enhancement](https://mlsanity.com/paper/2303.17569/) (2023)
- [CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes](https://mlsanity.com/paper/2304.06061/) (2023)
- [FlexHDR: Modelling Alignment and Exposure Uncertainties for Flexible HDR Imaging](https://mlsanity.com/paper/2201.02625/) (2022)
- [CLIP-Guided StyleGAN Inversion for Text-Driven Real Image Editing](https://mlsanity.com/paper/2307.08397/) (2023)

## Built on by (most cited first, 11 of 11)

- [UnfoldIR: Rethinking Deep Unfolding Network in Illumination Degradation Image Restoration](https://mlsanity.com/paper/2505.06683/) (2025)
- [Beyond Ground-Truth: Leveraging Image Quality Priors for Real-World Image Restoration](https://mlsanity.com/paper/2603.29773/) (2026)
- [VTD-CLIP: Video-to-Text Discretization via Prompting CLIP](https://mlsanity.com/paper/2503.18407/) (2025)
- [DINOLight: Robust Ambient Light Normalization with Self-supervised Visual Prior Integration](https://mlsanity.com/paper/2603.12579/) (2026)
- [DIME-Net: A Dual-Illumination Adaptive Enhancement Network Based on Retinex and Mixture-of-Experts](https://mlsanity.com/paper/2508.13921/) (2025)
- [SportSkills: Physical Skill Learning from Sports Instructional Videos](https://mlsanity.com/paper/2603.25163/) (2026)
- [RestoRect: Degraded Image Restoration via Latent Rectified Flow & Feature Distillation](https://mlsanity.com/paper/2509.23480/) (2025)
- [UnfoldLDM: Degradation-Aware Unfolding with Iterative Latent Diffusion Priors for Blind Image Restoration](https://mlsanity.com/paper/2511.18152/) (2025)
- [Retinex Meets Language: A Physics-Semantics-Guided Underwater Image Enhancement Network](https://mlsanity.com/paper/2603.07076/) (2026)
- [ZeroIDIR: Zero-Reference Illumination Degradation Image Restoration with Perturbed Consistency Diffusion Models](https://mlsanity.com/paper/2605.11435/) (2026)
- [UBLLIE: Unified Backlight and Low-Light Image Enhancement](https://mlsanity.com/paper/2608.04429/) (2026)

## Abstract (arXiv)

In this paper we propose a novel modification of Contrastive Language-Image Pre-Training (CLIP) guidance for the task of unsupervised backlit image enhancement. Our work builds on the state-of-the-art CLIP-LIT approach, which learns a prompt pair by constraining the text-image similarity between a prompt (negative/positive sample) and a corresponding image (backlit image/well-lit image) in the CLIP embedding space. Learned prompts then guide an image enhancement network. Based on the CLIP-LIT framework, we propose two novel methods for CLIP guidance. First, we show that instead of tuning prompts in the space of text embeddings, it is possible to directly tune their embeddings in the latent space without any loss in quality. This accelerates training and potentially enables the use of additional encoders that do not have a text encoder. Second, we propose a novel approach that does not require any prompt tuning. Instead, based on CLIP embeddings of backlit and well-lit images from training data, we compute the residual vector in the embedding space as a simple difference between the mean embeddings of the well-lit and backlit images. This vector then guides the enhancement network during training, pushing a backlit image towards the space of well-lit images. This approach further dramatically reduces training time, stabilizes training and produces high quality enhanced images without artifacts, both in supervised and unsupervised training regimes. Additionally, we show that residual vectors can be interpreted, revealing biases in training data, and thereby enabling potential bias correction.