Statistical Learning Theory of Quasi-Regular Cases

Koshi YAMADA; Sumio WATANABE

doi:10.1587/transfun.E95.A.2479

Statistical Learning Theory of Quasi-Regular Cases

Koshi YAMADA, Sumio WATANABE

Full Text Views

0

Cite this

Summary :

Many learning machines such as normal mixtures and layered neural networks are not regular but singular statistical models, because the map from a parameter to a probability distribution is not one-to-one. The conventional statistical asymptotic theory can not be applied to such learning machines because the likelihood function can not be approximated by any normal distribution. Recently, new statistical theory has been established based on algebraic geometry and it was clarified that the generalization and training errors are determined by two birational invariants, the real log canonical threshold and the singular fluctuation. However, their concrete values are left unknown. In the present paper, we propose a new concept, a quasi-regular case in statistical learning theory. A quasi-regular case is not a regular case but a singular case, however, it has the same property as a regular case. In fact, we prove that, in a quasi-regular case, two birational invariants are equal to each other, resulting that the symmetry of the generalization and training errors holds. Moreover, the concrete values of two birational invariants are explicitly obtained, hence the quasi-regular case is useful to study statistical learning theory.

Publication: IEICE TRANSACTIONS on Fundamentals Vol.E95-A No.12 pp.2479-2487

Publication Date: 2012/12/01

Publicized

Online ISSN: 1745-1337

DOI: 10.1587/transfun.E95.A.2479

Type of Manuscript: PAPER

Category: General Fundamentals and Boundaries

Cite this

Copy

Koshi YAMADA, Sumio WATANABE, "Statistical Learning Theory of Quasi-Regular Cases" in IEICE TRANSACTIONS on Fundamentals, vol. E95-A, no. 12, pp. 2479-2487, December 2012, doi: 10.1587/transfun.E95.A.2479.
Abstract: Many learning machines such as normal mixtures and layered neural networks are not regular but singular statistical models, because the map from a parameter to a probability distribution is not one-to-one. The conventional statistical asymptotic theory can not be applied to such learning machines because the likelihood function can not be approximated by any normal distribution. Recently, new statistical theory has been established based on algebraic geometry and it was clarified that the generalization and training errors are determined by two birational invariants, the real log canonical threshold and the singular fluctuation. However, their concrete values are left unknown. In the present paper, we propose a new concept, a quasi-regular case in statistical learning theory. A quasi-regular case is not a regular case but a singular case, however, it has the same property as a regular case. In fact, we prove that, in a quasi-regular case, two birational invariants are equal to each other, resulting that the symmetry of the generalization and training errors holds. Moreover, the concrete values of two birational invariants are explicitly obtained, hence the quasi-regular case is useful to study statistical learning theory.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.E95.A.2479/_p

Copy

@ARTICLE{e95-a_12_2479,
author={Koshi YAMADA, Sumio WATANABE, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Statistical Learning Theory of Quasi-Regular Cases},
year={2012},
volume={E95-A},
number={12},
pages={2479-2487},
abstract={Many learning machines such as normal mixtures and layered neural networks are not regular but singular statistical models, because the map from a parameter to a probability distribution is not one-to-one. The conventional statistical asymptotic theory can not be applied to such learning machines because the likelihood function can not be approximated by any normal distribution. Recently, new statistical theory has been established based on algebraic geometry and it was clarified that the generalization and training errors are determined by two birational invariants, the real log canonical threshold and the singular fluctuation. However, their concrete values are left unknown. In the present paper, we propose a new concept, a quasi-regular case in statistical learning theory. A quasi-regular case is not a regular case but a singular case, however, it has the same property as a regular case. In fact, we prove that, in a quasi-regular case, two birational invariants are equal to each other, resulting that the symmetry of the generalization and training errors holds. Moreover, the concrete values of two birational invariants are explicitly obtained, hence the quasi-regular case is useful to study statistical learning theory.},
keywords={},
doi={10.1587/transfun.E95.A.2479},
ISSN={1745-1337},
month={December},}

Copy

TY - JOUR
TI - Statistical Learning Theory of Quasi-Regular Cases
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 2479
EP - 2487
AU - Koshi YAMADA
AU - Sumio WATANABE
PY - 2012
DO - 10.1587/transfun.E95.A.2479
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E95-A
IS - 12
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - December 2012
AB - Many learning machines such as normal mixtures and layered neural networks are not regular but singular statistical models, because the map from a parameter to a probability distribution is not one-to-one. The conventional statistical asymptotic theory can not be applied to such learning machines because the likelihood function can not be approximated by any normal distribution. Recently, new statistical theory has been established based on algebraic geometry and it was clarified that the generalization and training errors are determined by two birational invariants, the real log canonical threshold and the singular fluctuation. However, their concrete values are left unknown. In the present paper, we propose a new concept, a quasi-regular case in statistical learning theory. A quasi-regular case is not a regular case but a singular case, however, it has the same property as a regular case. In fact, we prove that, in a quasi-regular case, two birational invariants are equal to each other, resulting that the symmetry of the generalization and training errors holds. Moreover, the concrete values of two birational invariants are explicitly obtained, hence the quasi-regular case is useful to study statistical learning theory.
ER -