# Job recommendations: benchmarking of collaborative filtering methods for classifieds

arXiv: https://arxiv.org/abs/2301.07946 · HTML: https://mlsanity.com/paper/2301.07946/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 78 references that are in the map.
- First posted on arXiv 19 Jan 2023.

## Builds on (most cited first, 13 of 13)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Item2Vec: Neural Item Embedding for Collaborative Filtering](https://mlsanity.com/paper/1603.04259/) (2016)
- [A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research](https://mlsanity.com/paper/1911.07698/) (2019)
- [BARS: Towards Open Benchmarking for Recommender Systems](https://mlsanity.com/paper/2205.09626/) (2022)
- [Exploring Data Splitting Strategies for the Evaluation of Recommendation Models](https://mlsanity.com/paper/2007.13237/) (2020)
- [Metadata Embeddings for User and Item Cold-start Recommendations](https://mlsanity.com/paper/1507.08439/) (2015)
- [E-commerce in Your Inbox: Product Recommendations at Scale](https://mlsanity.com/paper/1606.07154/) (2016)
- [Quality Metrics in Recommender Systems: Do We Calculate Metrics Consistently?](https://mlsanity.com/paper/2206.12858/) (2022)
- [Deep Job Understanding at LinkedIn](https://mlsanity.com/paper/2006.12425/) (2020)
- [Top-N Recommendation Algorithms: A Quest for the State-of-the-Art](https://mlsanity.com/paper/2203.01155/) (2022)
- [Reenvisioning Collaborative Filtering vs Matrix Factorization](https://mlsanity.com/paper/2107.13472/) (2021)

## Abstract (arXiv)

Classifieds provide many challenges for recommendation methods, due to the limited information regarding users and items. In this paper, we explore recommendation methods for classifieds using the example of OLX Jobs. The goal of the paper is to benchmark different recommendation methods for jobs classifieds in order to improve advertisements' conversion rate and user satisfaction. In our research, we implemented methods that are scalable and represent different approaches to recommendation, namely ALS, LightFM, Prod2Vec, RP3beta, and SLIM. We performed a laboratory comparison of methods with regard to accuracy, diversity, and scalability (memory and time consumption during training and in prediction). Online A/B tests were also carried out by sending millions of messages with recommendations to evaluate models in a real-world setting. In addition, we have published the dataset that we created for the needs of our research. To the best of our knowledge, this is the first dataset of this kind. The dataset contains 65,502,201 events performed on OLX Jobs by 3,295,942 users, who interacted with (displayed, replied to, or bookmarked) 185,395 job ads in two weeks of 2020. We demonstrate that RP3beta, SLIM, and ALS perform significantly better than Prod2Vec and LightFM when tested in a laboratory setting. Online A/B tests also demonstrated that sending messages with recommendations generated by the ALS and RP3beta models increases the number of users contacting advertisers. Additionally, RP3beta had a 20% greater impact on this metric than ALS.