# Meta-Learning for Relative Density-Ratio Estimation

arXiv: https://arxiv.org/abs/2107.00801 · HTML: https://mlsanity.com/paper/2107.00801/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 69 references that are in the map.
- First posted on arXiv 2 Jul 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Meta-Learning with Implicit Gradients](https://mlsanity.com/paper/1909.04630/) (2019)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Meta-Learning: A Survey](https://mlsanity.com/paper/1810.03548/) (2018)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [Positive-Unlabeled Learning with Non-Negative Risk Estimator](https://mlsanity.com/paper/1703.00593/) (2017)
- [Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks](https://mlsanity.com/paper/1810.00825/) (2018)
- [Telescoping Density-Ratio Estimation](https://mlsanity.com/paper/2006.12204/) (2020)
- [Rethinking Importance Weighting for Deep Learning under Distribution Shift](https://mlsanity.com/paper/2006.04662/) (2020)
- [Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation](https://mlsanity.com/paper/2006.06979/) (2020)
- [Variational Autoencoder with Implicit Optimal Priors](https://mlsanity.com/paper/1809.05284/) (2018)
- [Generative Adversarial Nets from a Density Ratio Estimation Perspective](https://mlsanity.com/paper/1610.02920/) (2016)
- [Trimmed Density Ratio Estimation](https://mlsanity.com/paper/1703.03216/) (2017)
- [Self-Paced Multi-Task Learning](https://mlsanity.com/paper/1604.01474/) (2016)

## Built on by (most cited first, 6 of 6)

- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [Meta-learning for Positive-unlabeled Classification](https://mlsanity.com/paper/2406.03680/) (2024)
- [Unified Perspective on Probability Divergence via Maximum Likelihood Density Ratio Estimation: Bridging KL-Divergence and Integral Probability Metrics](https://mlsanity.com/paper/2201.13127/) (2022)
- [Some notes concerning a generalized KMM-type optimization method for density ratio estimation](https://mlsanity.com/paper/2309.07887/) (2023)
- [Relative Density Ratio Optimization for Stable and Statistically Consistent Model Alignment](https://mlsanity.com/paper/2604.04410/) (2026)
- [Importance Weighting for Unlabeled-unlabeled Learning under Distribution Shift](https://mlsanity.com/paper/2609.10994/) (2026)

## Abstract (arXiv)

The ratio of two probability densities, called a density-ratio, is a vital quantity in machine learning. In particular, a relative density-ratio, which is a bounded extension of the density-ratio, has received much attention due to its stability and has been used in various applications such as outlier detection and dataset comparison. Existing methods for (relative) density-ratio estimation (DRE) require many instances from both densities. However, sufficient instances are often unavailable in practice. In this paper, we propose a meta-learning method for relative DRE, which estimates the relative density-ratio from a few instances by using knowledge in related datasets. Specifically, given two datasets that consist of a few instances, our model extracts the datasets' information by using neural networks and uses it to obtain instance embeddings appropriate for the relative DRE. We model the relative density-ratio by a linear model on the embedded space, whose global optimum solution can be obtained as a closed-form solution. The closed-form solution enables fast and effective adaptation to a few instances, and its differentiability enables us to train our model such that the expected test error for relative DRE can be explicitly minimized after adapting to a few instances. We empirically demonstrate the effectiveness of the proposed method by using three problems: relative DRE, dataset comparison, and outlier detection.