The search functionality is under construction.

The search functionality is under construction.

Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical Kullback-Leibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.

- Publication
- IEICE TRANSACTIONS on Fundamentals Vol.E93-A No.4 pp.787-798

- Publication Date
- 2010/04/01

- Publicized

- Online ISSN
- 1745-1337

- DOI
- 10.1587/transfun.E93.A.787

- Type of Manuscript
- PAPER

- Category
- Algorithms and Data Structures

The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.

Copy

Takafumi KANAMORI, Taiji SUZUKI, Masashi SUGIYAMA, "Theoretical Analysis of Density Ratio Estimation" in IEICE TRANSACTIONS on Fundamentals,
vol. E93-A, no. 4, pp. 787-798, April 2010, doi: 10.1587/transfun.E93.A.787.

Abstract: Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical Kullback-Leibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.

URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.E93.A.787/_p

Copy

@ARTICLE{e93-a_4_787,

author={Takafumi KANAMORI, Taiji SUZUKI, Masashi SUGIYAMA, },

journal={IEICE TRANSACTIONS on Fundamentals},

title={Theoretical Analysis of Density Ratio Estimation},

year={2010},

volume={E93-A},

number={4},

pages={787-798},

abstract={Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical Kullback-Leibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.},

keywords={},

doi={10.1587/transfun.E93.A.787},

ISSN={1745-1337},

month={April},}

Copy

TY - JOUR

TI - Theoretical Analysis of Density Ratio Estimation

T2 - IEICE TRANSACTIONS on Fundamentals

SP - 787

EP - 798

AU - Takafumi KANAMORI

AU - Taiji SUZUKI

AU - Masashi SUGIYAMA

PY - 2010

DO - 10.1587/transfun.E93.A.787

JO - IEICE TRANSACTIONS on Fundamentals

SN - 1745-1337

VL - E93-A

IS - 4

JA - IEICE TRANSACTIONS on Fundamentals

Y1 - April 2010

AB - Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical Kullback-Leibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.

ER -