# PALATE: Personalized Aesthetic Learning through Adaptive Taste Evolution for Multi-User Portrait Retouching

arXiv: https://arxiv.org/abs/2608.18622 · HTML: https://mlsanity.com/paper/2608.18622/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023)
- [LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics](https://mlsanity.com/paper/2511.08544/) (2025)
- [Deep Bilateral Learning for Real-Time Image Enhancement](https://mlsanity.com/paper/1707.02880/) (2017)
- [HIVE: Harnessing Human Feedback for Instructional Visual Editing](https://mlsanity.com/paper/2303.09618/) (2023)
- [DeepLPF: Deep Local Parametric Filters for Image Enhancement](https://mlsanity.com/paper/2003.13985/) (2020)
- [Densely Guided Knowledge Distillation using Multiple Teacher Assistants](https://mlsanity.com/paper/2009.08825/) (2020)
- [Conditional Sequential Modulation for Efficient Global Image Retouching](https://mlsanity.com/paper/2009.10390/) (2020)
- [EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing](https://mlsanity.com/paper/2509.26346/) (2025)
- [PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency](https://mlsanity.com/paper/2105.09180/) (2021)
- [Personalized Image Aesthetics Assessment with Rich Attributes](https://mlsanity.com/paper/2203.16754/) (2022)
- [LoRe: Personalizing LLMs via Low-Rank Reward Modeling](https://mlsanity.com/paper/2504.14439/) (2025)
- [Personalized Preference Fine-tuning of Diffusion Models](https://mlsanity.com/paper/2501.06655/) (2025)
- [Knowledge Evolution in Neural Networks](https://mlsanity.com/paper/2103.05152/) (2021)
- [Affective Image Editing: Shaping Emotional Factors via Text Descriptions](https://mlsanity.com/paper/2505.18699/) (2025)
- [Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors](https://mlsanity.com/paper/2512.17376/) (2025)
- [Fine-grained Image Aesthetic Assessment: Learning Discriminative Scores from Relative Ranks](https://mlsanity.com/paper/2603.03907/) (2026)
- [Personalized Image Filter: Mastering Your Photographic Style](https://mlsanity.com/paper/2510.16791/) (2025)
- [ORC: Network Group-based Knowledge Distillation using Online Role Change](https://mlsanity.com/paper/2206.01186/) (2022)

## Abstract (arXiv)

Automatic portrait retouching has advanced rapidly, yet its objective is inherently subjective: the same portrait admits multiple professionally valid results, and users disagree about which one is best. Most existing methods optimize a population-level aesthetic standard and therefore cannot capture individual taste, while fine-tuning a separate editing model for every user incurs prohibitive training, storage, and data costs. We propose PALATE, a shared reward-evolution framework that keeps the image editor fixed and instead personalizes the selection among retouched candidates of the same source portrait. PALATE decomposes the reward for each user into a global backbone shared by all users, category-level residuals shared by aesthetically similar users, and a lightweight user adapter, with anti-collapse regularizers keeping the three levels complementary.A cyclic dual-level distillation scheme first distills user-specific preferences into category rewards and then consolidates the resulting category-level knowledge into the global backbone, which is redistributed to initialize the next evolution round. In this way, the shared initialization improves progressively across rounds, enabling unseen users to be calibrated from only a few rankings. On expert-retouched candidates from PPR10K with held-out users and held-out images, PALATE attains 72.83% pairwise preference-prediction accuracy, surpassing all reward, aesthetic, and image-quality baselines, of which the strongest, PickScore, reaches 58.06%. Each new user costs only 512 bytes of user-specific parameters and millisecond-level scoring.