# CLIPtone: Unsupervised Learning for Text-based Image Tone Adjustment

arXiv: https://arxiv.org/abs/2404.01123 · HTML: https://mlsanity.com/paper/2404.01123/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 53 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 30 of 37)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance](https://mlsanity.com/paper/2204.08583/) (2022)
- [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks](https://mlsanity.com/paper/2106.04489/) (2021)
- [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021)
- [HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing](https://mlsanity.com/paper/2111.15666/) (2021)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [Deep Bilateral Learning for Real-Time Image Enhancement](https://mlsanity.com/paper/1707.02880/) (2017)
- [Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time](https://mlsanity.com/paper/2009.14468/) (2020)
- [DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks](https://mlsanity.com/paper/1704.02470/) (2017)
- [ManiGAN: Text-Guided Image Manipulation](https://mlsanity.com/paper/1912.06203/) (2019)
- [DeepLPF: Deep Local Parametric Filters for Image Enhancement](https://mlsanity.com/paper/2003.13985/) (2020)
- [Paint by Word](https://mlsanity.com/paper/2103.10951/) (2021)
- [Conditional Sequential Modulation for Efficient Global Image Retouching](https://mlsanity.com/paper/2009.10390/) (2020)
- [AdaInt: Learning Adaptive Intervals for 3D Lookup Tables on Real-time Image Enhancement](https://mlsanity.com/paper/2204.13983/) (2022)
- [Real-time Image Enhancer via Learnable Spatial-aware 3D Lookup Tables](https://mlsanity.com/paper/2108.08697/) (2021)
- [PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency](https://mlsanity.com/paper/2105.09180/) (2021)
- [GAN Inversion for Out-of-Range Images with Geometric Transformations](https://mlsanity.com/paper/2108.08998/) (2021)

## Built on by (most cited first, 10 of 10)

- [Luminance-GS: Adapting 3D Gaussian Splatting to Challenging Lighting Conditions with View-Adaptive Curve Adjustment](https://mlsanity.com/paper/2504.01503/) (2025)
- [Unifying Color and Lightness Correction with View-Adaptive Curve Adjustment for Robust 3D Novel View Synthesis](https://mlsanity.com/paper/2602.18322/) (2026)
- [Personalized Image Filter: Mastering Your Photographic Style](https://mlsanity.com/paper/2510.16791/) (2025)
- [AceTone: Bridging Words and Colors for Conditional Image Grading](https://mlsanity.com/paper/2604.00530/) (2026)
- [IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment](https://mlsanity.com/paper/2606.08016/) (2026)
- [CURVE: CLIP-Utilized Reinforcement Learning for Visual Image Enhancement via Simple Image Processing](https://mlsanity.com/paper/2505.23102/) (2025)
- [MRStyle: A Unified Framework for Color Style Transfer with Multi-Modality Reference](https://mlsanity.com/paper/2409.05250/) (2024)
- [COT-AD: Cotton Analysis Dataset](https://mlsanity.com/paper/2507.18532/) (2025)
- [Edge-Aware Image Manipulation via Diffusion Models with a Novel Structure-Preservation Loss](https://mlsanity.com/paper/2601.16645/) (2026)
- [GLUT: 3D Gaussian Lookup Table for Continuous Color Transformation](https://mlsanity.com/paper/2605.19889/) (2026)

## Abstract (arXiv)

Recent image tone adjustment (or enhancement) approaches have predominantly adopted supervised learning for learning human-centric perceptual assessment. However, these approaches are constrained by intrinsic challenges of supervised learning. Primarily, the requirement for expertly-curated or retouched images escalates the data acquisition expenses. Moreover, their coverage of target style is confined to stylistic variants inferred from the training data. To surmount the above challenges, we propose an unsupervised learning-based approach for text-based image tone adjustment method, CLIPtone, that extends an existing image enhancement method to accommodate natural language descriptions. Specifically, we design a hyper-network to adaptively modulate the pretrained parameters of the backbone model based on text description. To assess whether the adjusted image aligns with the text description without ground truth image, we utilize CLIP, which is trained on a vast set of language-image pairs and thus encompasses knowledge of human perception. The major advantages of our approach are three fold: (i) minimal data collection expenses, (ii) support for a range of adjustments, and (iii) the ability to handle novel text descriptions unseen in training. Our approach's efficacy is demonstrated through comprehensive experiments, including a user study.