# Integrity and Junkiness Failure Handling for Embedding-based Retrieval: A Case Study in Social Network Search

arXiv: https://arxiv.org/abs/2304.09287 · HTML: https://mlsanity.com/paper/2304.09287/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 10 references that are in the map.
- First posted on arXiv 18 Apr 2023.

## Builds on (most cited first, 8 of 8)

- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [Fairness in Deep Learning: A Computational Perspective](https://mlsanity.com/paper/1908.08843/) (2019)
- [User-oriented Fairness in Recommendation](https://mlsanity.com/paper/2104.10671/) (2021)
- [Semantic Product Search](https://mlsanity.com/paper/1907.00937/) (2019)
- [Embedding-based Product Retrieval in Taobao Search](https://mlsanity.com/paper/2106.09297/) (2021)
- [Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning](https://mlsanity.com/paper/2006.02282/) (2020)
- [Pre-trained Language Model for Web-scale Retrieval in Baidu Search](https://mlsanity.com/paper/2106.03373/) (2021)

## Built on by (most cited first, 2 of 2)

- [Multi-Modal Machine Learning for Assessing Gaming Skills in Online Streaming: A Case Study with CS:GO](https://mlsanity.com/paper/2307.12236/) (2023)
- [Semantic Recall for Vector Search](https://mlsanity.com/paper/2604.20417/) (2026)

## Abstract (arXiv)

Embedding based retrieval has seen its usage in a variety of search applications like e-commerce, social networking search etc. While the approach has demonstrated its efficacy in tasks like semantic matching and contextual search, it is plagued by the problem of uncontrollable relevance. In this paper, we conduct an analysis of embedding-based retrieval launched in early 2021 on our social network search engine, and define two main categories of failures introduced by it, integrity and junkiness. The former refers to issues such as hate speech and offensive content that can severely harm user experience, while the latter includes irrelevant results like fuzzy text matching or language mismatches. Efficient methods during model inference are further proposed to resolve the issue, including indexing treatments and targeted user cohort treatments, etc. Though being simple, we show the methods have good offline NDCG and online A/B tests metrics gain in practice. We analyze the reasons for the improvements, pointing out that our methods are only preliminary attempts to this important but challenging problem. We put forward potential future directions to explore.