Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical Kullback-Leibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copy
Takafumi KANAMORI, Taiji SUZUKI, Masashi SUGIYAMA, "Theoretical Analysis of Density Ratio Estimation" in IEICE TRANSACTIONS on Fundamentals,
vol. E93-A, no. 4, pp. 787-798, April 2010, doi: 10.1587/transfun.E93.A.787.
Abstract: Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical Kullback-Leibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.E93.A.787/_p
Copy
@ARTICLE{e93-a_4_787,
author={Takafumi KANAMORI, Taiji SUZUKI, Masashi SUGIYAMA, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Theoretical Analysis of Density Ratio Estimation},
year={2010},
volume={E93-A},
number={4},
pages={787-798},
abstract={Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical Kullback-Leibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.},
keywords={},
doi={10.1587/transfun.E93.A.787},
ISSN={1745-1337},
month={April},}
Copy
TY - JOUR
TI - Theoretical Analysis of Density Ratio Estimation
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 787
EP - 798
AU - Takafumi KANAMORI
AU - Taiji SUZUKI
AU - Masashi SUGIYAMA
PY - 2010
DO - 10.1587/transfun.E93.A.787
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E93-A
IS - 4
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - April 2010
AB - Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical Kullback-Leibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.
ER -