# Zero-Shot Recommendations with Pre-Trained Large Language Models for Multimodal Nudging

arXiv: https://arxiv.org/abs/2309.01026 · HTML: https://mlsanity.com/paper/2309.01026/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 75 references that are in the map.
- First posted on arXiv 2 Sep 2023.

## Builds on (most cited first, 19 of 19)

- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [A Review of Generalized Zero-Shot Learning Methods](https://mlsanity.com/paper/2011.08641/) (2020)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [RecSim: A Configurable Simulation Platform for Recommender Systems](https://mlsanity.com/paper/1909.04847/) (2019)
- [RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising](https://mlsanity.com/paper/1808.00720/) (2018)
- [Zero-Shot Next-Item Recommendation using Large Pretrained Language Models](https://mlsanity.com/paper/2304.03153/) (2023)
- [Zero-Shot Recommender Systems](https://mlsanity.com/paper/2105.08318/) (2021)
- [Multimodal Recommender Systems: A Survey](https://mlsanity.com/paper/2302.03883/) (2023)
- [From Zero-Shot Learning to Cold-Start Recommendation](https://mlsanity.com/paper/1906.08511/) (2019)
- [The Use of Machine Learning Algorithms in Recommender Systems: A Systematic Review](https://mlsanity.com/paper/1511.05263/) (2015)
- [Digital Nudging with Recommender Systems: Survey and Future Directions](https://mlsanity.com/paper/2011.03413/) (2020)
- [A Comprehensive Survey on Multimodal Recommender Systems: Taxonomy, Evaluation, and Future Directions](https://mlsanity.com/paper/2302.04473/) (2023)
- [Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventions](https://mlsanity.com/paper/1802.09714/) (2018)
- [A clustering-based reinforcement learning approach for tailored personalization of e-Health interventions](https://mlsanity.com/paper/1804.03592/) (2018)
- [MM-Rec: Multimodal News Recommendation](https://mlsanity.com/paper/2104.07407/) (2021)
- [On the Unreasonable Efficiency of State Space Clustering in Personalization Tasks](https://mlsanity.com/paper/2112.13141/) (2021)
- [Simulated Contextual Bandits for Personalization Tasks from Recommendation Datasets](https://mlsanity.com/paper/2210.10631/) (2022)

## Built on by (most cited first, 4 of 4)

- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guide](https://mlsanity.com/paper/2402.14837/) (2024)
- [Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey](https://mlsanity.com/paper/2312.06718/) (2023)
- [Driver Assistant: Persuading Drivers to Adjust Secondary Tasks Using Large Language Models](https://mlsanity.com/paper/2508.05238/) (2025)

## Abstract (arXiv)

We present a method for zero-shot recommendation of multimodal non-stationary content that leverages recent advancements in the field of generative AI. We propose rendering inputs of different modalities as textual descriptions and to utilize pre-trained LLMs to obtain their numerical representations by computing semantic embeddings. Once unified representations of all content items are obtained, the recommendation can be performed by computing an appropriate similarity metric between them without any additional learning. We demonstrate our approach on a synthetic multimodal nudging environment, where the inputs consist of tabular, textual, and visual data.