Constrained Least-Squares Density-Difference Estimation
Summary :
We address the problem of estimating the difference between two probability densities. A naive approach is a two-step procedure that first estimates two densities separately and then computes their difference. However, such a two-step procedure does not necessarily work well because the first step is performed without regard to the second step and thus a small error in the first stage can cause a big error in the second stage. Recently, a single-shot method called the least-squares density-difference (LSDD) estimator has been proposed. LSDD directly estimates the density difference without separately estimating two densities, and it was demonstrated to outperform the two-step approach. In this paper, we propose a variation of LSDD called the constrained least-squares density-difference (CLSDD) estimator, and theoretically prove that CLSDD improves the accuracy of density difference estimation for correctly specified parametric models. The usefulness of the proposed method is also demonstrated experimentally.
- Publication
- IEICE TRANSACTIONS on Information Vol.E97-D No.7 pp.1822-1829
- Publication Date
- 2014/07/01
- Publicized
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.E97.D.1822
- Type of Manuscript
- PAPER
- Category
- Artificial Intelligence, Data Mining
Authors
Tuan Duong NGUYEN
Tokyo Institute of Technology
Marthinus Christoffel DU PLESSIS
Tokyo Institute of Technology
Takafumi KANAMORI
Nagoya University
Masashi SUGIYAMA
Tokyo Institute of Technology
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Tuan Duong NGUYEN, Marthinus Christoffel DU PLESSIS, Takafumi KANAMORI, Masashi SUGIYAMA, "Constrained Least-Squares Density-Difference Estimation" in IEICE TRANSACTIONS on Information,
vol. E97-D, no. 7, pp. 1822-1829, July 2014, doi: 10.1587/transinf.E97.D.1822.
Abstract: We address the problem of estimating the difference between two probability densities. A naive approach is a two-step procedure that first estimates two densities separately and then computes their difference. However, such a two-step procedure does not necessarily work well because the first step is performed without regard to the second step and thus a small error in the first stage can cause a big error in the second stage. Recently, a single-shot method called the least-squares density-difference (LSDD) estimator has been proposed. LSDD directly estimates the density difference without separately estimating two densities, and it was demonstrated to outperform the two-step approach. In this paper, we propose a variation of LSDD called the constrained least-squares density-difference (CLSDD) estimator, and theoretically prove that CLSDD improves the accuracy of density difference estimation for correctly specified parametric models. The usefulness of the proposed method is also demonstrated experimentally.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.E97.D.1822/_p
Copy
@ARTICLE{e97-d_7_1822,
author={Tuan Duong NGUYEN, Marthinus Christoffel DU PLESSIS, Takafumi KANAMORI, Masashi SUGIYAMA, },
journal={IEICE TRANSACTIONS on Information},
title={Constrained Least-Squares Density-Difference Estimation},
year={2014},
volume={E97-D},
number={7},
pages={1822-1829},
abstract={We address the problem of estimating the difference between two probability densities. A naive approach is a two-step procedure that first estimates two densities separately and then computes their difference. However, such a two-step procedure does not necessarily work well because the first step is performed without regard to the second step and thus a small error in the first stage can cause a big error in the second stage. Recently, a single-shot method called the least-squares density-difference (LSDD) estimator has been proposed. LSDD directly estimates the density difference without separately estimating two densities, and it was demonstrated to outperform the two-step approach. In this paper, we propose a variation of LSDD called the constrained least-squares density-difference (CLSDD) estimator, and theoretically prove that CLSDD improves the accuracy of density difference estimation for correctly specified parametric models. The usefulness of the proposed method is also demonstrated experimentally.},
keywords={},
doi={10.1587/transinf.E97.D.1822},
ISSN={1745-1361},
month={July},}
Copy
TY - JOUR
TI - Constrained Least-Squares Density-Difference Estimation
T2 - IEICE TRANSACTIONS on Information
SP - 1822
EP - 1829
AU - Tuan Duong NGUYEN
AU - Marthinus Christoffel DU PLESSIS
AU - Takafumi KANAMORI
AU - Masashi SUGIYAMA
PY - 2014
DO - 10.1587/transinf.E97.D.1822
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E97-D
IS - 7
JA - IEICE TRANSACTIONS on Information
Y1 - July 2014
AB - We address the problem of estimating the difference between two probability densities. A naive approach is a two-step procedure that first estimates two densities separately and then computes their difference. However, such a two-step procedure does not necessarily work well because the first step is performed without regard to the second step and thus a small error in the first stage can cause a big error in the second stage. Recently, a single-shot method called the least-squares density-difference (LSDD) estimator has been proposed. LSDD directly estimates the density difference without separately estimating two densities, and it was demonstrated to outperform the two-step approach. In this paper, we propose a variation of LSDD called the constrained least-squares density-difference (CLSDD) estimator, and theoretically prove that CLSDD improves the accuracy of density difference estimation for correctly specified parametric models. The usefulness of the proposed method is also demonstrated experimentally.
ER -
English
