IEICE TRANSACTIONS on Information
Computationally Efficient Estimation of Squared-Loss Mutual Information with Multiplicative Kernel Models
Summary :
Squared-loss mutual information (SMI) is a robust measure of the statistical dependence between random variables. The sample-based SMI approximator called least-squares mutual information (LSMI) was demonstrated to be useful in performing various machine learning tasks such as dimension reduction, clustering, and causal inference. The original LSMI approximates the pointwise mutual information by using the kernel model, which is a linear combination of kernel basis functions located on paired data samples. Although LSMI was proved to achieve the optimal approximation accuracy asymptotically, its approximation capability is limited when the sample size is small due to an insufficient number of kernel basis functions. Increasing the number of kernel basis functions can mitigate this weakness, but a naive implementation of this idea significantly increases the computation costs. In this article, we show that the computational complexity of LSMI with the multiplicative kernel model, which locates kernel basis functions on unpaired data samples and thus the number of kernel basis functions is the sample size squared, is the same as that for the plain kernel model. We experimentally demonstrate that LSMI with the multiplicative kernel model is more accurate than that with plain kernel models in small sample cases, with only mild increase in computation time.
- Publication
- IEICE TRANSACTIONS on Information Vol.E97-D No.4 pp.968-971
- Publication Date
- 2014/04/01
- Publicized
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.E97.D.968
- Type of Manuscript
- LETTER
- Category
- Fundamentals of Information Systems
Authors
Tomoya SAKAI
Tokyo Institute of Technology
Masashi SUGIYAMA
Tokyo Institute of Technology
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Tomoya SAKAI, Masashi SUGIYAMA, "Computationally Efficient Estimation of Squared-Loss Mutual Information with Multiplicative Kernel Models" in IEICE TRANSACTIONS on Information,
vol. E97-D, no. 4, pp. 968-971, April 2014, doi: 10.1587/transinf.E97.D.968.
Abstract: Squared-loss mutual information (SMI) is a robust measure of the statistical dependence between random variables. The sample-based SMI approximator called least-squares mutual information (LSMI) was demonstrated to be useful in performing various machine learning tasks such as dimension reduction, clustering, and causal inference. The original LSMI approximates the pointwise mutual information by using the kernel model, which is a linear combination of kernel basis functions located on paired data samples. Although LSMI was proved to achieve the optimal approximation accuracy asymptotically, its approximation capability is limited when the sample size is small due to an insufficient number of kernel basis functions. Increasing the number of kernel basis functions can mitigate this weakness, but a naive implementation of this idea significantly increases the computation costs. In this article, we show that the computational complexity of LSMI with the multiplicative kernel model, which locates kernel basis functions on unpaired data samples and thus the number of kernel basis functions is the sample size squared, is the same as that for the plain kernel model. We experimentally demonstrate that LSMI with the multiplicative kernel model is more accurate than that with plain kernel models in small sample cases, with only mild increase in computation time.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.E97.D.968/_p
Copy
@ARTICLE{e97-d_4_968,
author={Tomoya SAKAI, Masashi SUGIYAMA, },
journal={IEICE TRANSACTIONS on Information},
title={Computationally Efficient Estimation of Squared-Loss Mutual Information with Multiplicative Kernel Models},
year={2014},
volume={E97-D},
number={4},
pages={968-971},
abstract={Squared-loss mutual information (SMI) is a robust measure of the statistical dependence between random variables. The sample-based SMI approximator called least-squares mutual information (LSMI) was demonstrated to be useful in performing various machine learning tasks such as dimension reduction, clustering, and causal inference. The original LSMI approximates the pointwise mutual information by using the kernel model, which is a linear combination of kernel basis functions located on paired data samples. Although LSMI was proved to achieve the optimal approximation accuracy asymptotically, its approximation capability is limited when the sample size is small due to an insufficient number of kernel basis functions. Increasing the number of kernel basis functions can mitigate this weakness, but a naive implementation of this idea significantly increases the computation costs. In this article, we show that the computational complexity of LSMI with the multiplicative kernel model, which locates kernel basis functions on unpaired data samples and thus the number of kernel basis functions is the sample size squared, is the same as that for the plain kernel model. We experimentally demonstrate that LSMI with the multiplicative kernel model is more accurate than that with plain kernel models in small sample cases, with only mild increase in computation time.},
keywords={},
doi={10.1587/transinf.E97.D.968},
ISSN={1745-1361},
month={April},}
Copy
TY - JOUR
TI - Computationally Efficient Estimation of Squared-Loss Mutual Information with Multiplicative Kernel Models
T2 - IEICE TRANSACTIONS on Information
SP - 968
EP - 971
AU - Tomoya SAKAI
AU - Masashi SUGIYAMA
PY - 2014
DO - 10.1587/transinf.E97.D.968
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E97-D
IS - 4
JA - IEICE TRANSACTIONS on Information
Y1 - April 2014
AB - Squared-loss mutual information (SMI) is a robust measure of the statistical dependence between random variables. The sample-based SMI approximator called least-squares mutual information (LSMI) was demonstrated to be useful in performing various machine learning tasks such as dimension reduction, clustering, and causal inference. The original LSMI approximates the pointwise mutual information by using the kernel model, which is a linear combination of kernel basis functions located on paired data samples. Although LSMI was proved to achieve the optimal approximation accuracy asymptotically, its approximation capability is limited when the sample size is small due to an insufficient number of kernel basis functions. Increasing the number of kernel basis functions can mitigate this weakness, but a naive implementation of this idea significantly increases the computation costs. In this article, we show that the computational complexity of LSMI with the multiplicative kernel model, which locates kernel basis functions on unpaired data samples and thus the number of kernel basis functions is the sample size squared, is the same as that for the plain kernel model. We experimentally demonstrate that LSMI with the multiplicative kernel model is more accurate than that with plain kernel models in small sample cases, with only mild increase in computation time.
ER -
English
