# Retrieving Black-box Optimal Images from External Databases

arXiv: https://arxiv.org/abs/2112.14921 · HTML: https://mlsanity.com/paper/2112.14921/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 64 references that are in the map.
- First posted on arXiv 30 Dec 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [XGNN: Towards Model-Level Explanations of Graph Neural Networks](https://mlsanity.com/paper/2006.02587/) (2020)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [Synthesizing the preferred inputs for neurons in neural networks via deep generator networks](https://mlsanity.com/paper/1605.09304/) (2016)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [Conversational Recommender System](https://mlsanity.com/paper/1806.03277/) (2018)
- [Combinatorial Bayesian Optimization using the Graph Cartesian Product](https://mlsanity.com/paper/1902.00448/) (2019)
- [Bayesian Optimization of Combinatorial Structures](https://mlsanity.com/paper/1806.08838/) (2018)
- [Aggregating Deep Convolutional Features for Image Retrieval](https://mlsanity.com/paper/1510.07493/) (2015)
- [Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms](https://mlsanity.com/paper/1805.09843/) (2018)
- [Simultaneous Feature Learning and Hash Coding with Deep Neural Networks](https://mlsanity.com/paper/1504.03410/) (2015)
- [From Bandits to Experts: A Tale of Domination and Independence](https://mlsanity.com/paper/1307.4564/) (2013)
- [Contextual Semibandits via Supervised Learning Oracles](https://mlsanity.com/paper/1502.05890/) (2015)
- [Private Recommender Systems: How Can Users Build Their Own Fair Recommender Systems without Log Data?](https://mlsanity.com/paper/2105.12353/) (2021)
- [Bootstrapping Domain-Specific Content Discovery on the Web](https://mlsanity.com/paper/1902.09667/) (2019)

## Built on by (most cited first, 6 of 6)

- [Embarrassingly Simple Text Watermarks](https://mlsanity.com/paper/2310.08920/) (2023)
- [Making Translators Privacy-aware on the User's Side](https://mlsanity.com/paper/2312.04068/) (2023)
- [Overhead-free User-side Recommender Systems](https://mlsanity.com/paper/2411.07589/) (2024)
- [Solving the Cold Start Problem on One's Own as an End User via Preference Transfer](https://mlsanity.com/paper/2502.12398/) (2025)
- [Active Learning from the Web](https://mlsanity.com/paper/2210.08205/) (2022)
- [AgentRecommender: LLM Agents Enable Customizable Recommender Systems on the User Side](https://mlsanity.com/paper/2609.31166/) (2026)

## Abstract (arXiv)

Suppose we have a black-box function (e.g., deep neural network) that takes an image as input and outputs a value that indicates preference. How can we retrieve optimal images with respect to this function from an external database on the Internet? Standard retrieval problems in the literature (e.g., item recommendations) assume that an algorithm has full access to the set of items. In other words, such algorithms are designed for service providers. In this paper, we consider the retrieval problem under different assumptions. Specifically, we consider how users with limited access to an image database can retrieve images using their own black-box functions. This formulation enables a flexible and finer-grained image search defined by each user. We assume the user can access the database through a search query with tight API limits. Therefore, a user needs to efficiently retrieve optimal images in terms of the number of queries. We propose an efficient retrieval algorithm Tiara for this problem. In the experiments, we confirm that our proposed method performs better than several baselines under various settings.