# On Sampling Strategies for Neural Network-based Collaborative Filtering

arXiv: https://arxiv.org/abs/1706.07881 · HTML: https://mlsanity.com/paper/1706.07881/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2017.

## Built on by (most cited first, 30 of 71)

- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation](https://mlsanity.com/paper/2202.03104/) (2022)
- [CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP](https://mlsanity.com/paper/2110.11316/) (2021)
- [A Statutory Article Retrieval Dataset in French](https://mlsanity.com/paper/2108.11792/) (2021)
- [Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2309.17050/) (2023)
- [UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining](https://mlsanity.com/paper/2202.13469/) (2022)
- [PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings](https://mlsanity.com/paper/2201.12093/) (2022)
- [Self-supervised Product Quantization for Deep Unsupervised Image Retrieval](https://mlsanity.com/paper/2109.02244/) (2021)
- [IMF: Interactive Multimodal Fusion Model for Link Prediction](https://mlsanity.com/paper/2303.10816/) (2023)
- [RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank](https://mlsanity.com/paper/2305.16726/) (2023)
- [E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation](https://mlsanity.com/paper/2205.14912/) (2022)
- [On the Theories Behind Hard Negative Sampling for Recommendation](https://mlsanity.com/paper/2302.03472/) (2023)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling](https://mlsanity.com/paper/2206.11549/) (2022)
- [Fine-grained Category Discovery under Coarse-grained supervision with Hierarchical Weighted Self-contrastive Learning](https://mlsanity.com/paper/2210.07733/) (2022)
- [Don't Judge a Language Model by Its Last Layer: Contrastive Learning with Layer-Wise Attention Pooling](https://mlsanity.com/paper/2209.05972/) (2022)
- [Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces](https://mlsanity.com/paper/2105.05736/) (2021)
- [Imputing Out-of-Vocabulary Embeddings with LOVE Makes Language Models Robust with Little Cost](https://mlsanity.com/paper/2203.07860/) (2022)
- [Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networks](https://mlsanity.com/paper/2301.12847/) (2023)
- [Improved Diversity-Promoting Collaborative Metric Learning for Recommendation](https://mlsanity.com/paper/2409.01012/) (2024)
- [Recommender systems based on graph embedding techniques: A comprehensive review](https://mlsanity.com/paper/2109.09587/) (2021)
- [On Sampling Collaborative Filtering Datasets](https://mlsanity.com/paper/2201.04768/) (2022)
- [Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability](https://mlsanity.com/paper/2209.13262/) (2022)
- [Toward a Better Understanding of Loss Functions for Collaborative Filtering](https://mlsanity.com/paper/2308.06091/) (2023)
- [Multi-Sample based Contrastive Loss for Top-k Recommendation](https://mlsanity.com/paper/2109.00217/) (2021)
- [Enhancing Cognitive Diagnosis using Un-interacted Exercises: A Collaboration-aware Mixed Sampling Approach](https://mlsanity.com/paper/2312.10110/) (2023)
- [Contrastive Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2102.10557/) (2021)
- [FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning](https://mlsanity.com/paper/2209.06692/) (2022)
- [Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation](https://mlsanity.com/paper/2210.09773/) (2022)

## Abstract (arXiv)

Recent advances in neural networks have inspired people to design hybrid recommendation algorithms that can incorporate both (1) user-item interaction information and (2) content information including image, audio, and text. Despite their promising results, neural network-based recommendation algorithms pose extensive computational costs, making it challenging to scale and improve upon. In this paper, we propose a general neural network-based recommendation framework, which subsumes several existing state-of-the-art recommendation algorithms, and address the efficiency issue by investigating sampling strategies in the stochastic gradient descent training for the framework. We tackle this issue by first establishing a connection between the loss functions and the user-item interaction bipartite graph, where the loss function terms are defined on links while major computation burdens are located at nodes. We call this type of loss functions "graph-based" loss functions, for which varied mini-batch sampling strategies can have different computational costs. Based on the insight, three novel sampling strategies are proposed, which can significantly improve the training efficiency of the proposed framework (up to × 30 times speedup in our experiments), as well as improving the recommendation performance. Theoretical analysis is also provided for both the computational cost and the convergence. We believe the study of sampling strategies have further implications on general graph-based loss functions, and would also enable more research under the neural network-based recommendation framework.