# Beauty Beyond Words: Explainable Beauty Product Recommendations Using Ingredient-Based Product Attributes

arXiv: https://arxiv.org/abs/2409.13628 · HTML: https://mlsanity.com/paper/2409.13628/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]](https://mlsanity.com/paper/1806.01264/) (2018)
- [AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding](https://mlsanity.com/paper/2106.02318/) (2021)
- [Product Characterisation towards Personalisation: Learning Attributes from Unstructured Data to Recommend Fashion Products](https://mlsanity.com/paper/1803.07679/) (2018)
- [Improving the Accuracy of Beauty Product Recommendations by Assessing Face Illumination Quality](https://mlsanity.com/paper/2309.04022/) (2023)
- [Automated Material Properties Extraction For Enhanced Beauty Product Discovery and Makeup Virtual Try-on](https://mlsanity.com/paper/2312.00766/) (2023)

## Built on by (most cited first, 2 of 2)

- [Interacting Large Language Model Agents. Interpretable Models and Social Learning](https://mlsanity.com/paper/2411.01271/) (2024)
- [Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation](https://mlsanity.com/paper/2609.05363/) (2026)

## Abstract (arXiv)

Accurate attribute extraction is critical for beauty product recommendations and building trust with customers. This remains an open problem, as existing solutions are often unreliable and incomplete. We present a system to extract beauty-specific attributes using end-to-end supervised learning based on beauty product ingredients. A key insight to our system is a novel energy-based implicit model architecture. We show that this implicit model architecture offers significant benefits in terms of accuracy, explainability, robustness, and flexibility. Furthermore, our implicit model can be easily fine-tuned to incorporate additional attributes as they become available, making it more useful in real-world applications. We validate our model on a major e-commerce skincare product catalog dataset and demonstrate its effectiveness. Finally, we showcase how ingredient-based attribute extraction contributes to enhancing the explainability of beauty recommendations.