Almost Sure and Mean Convergence of Extended Stochastic Complexity

Masayuki GOTOH; Toshiyasu MATSUSHIMA; Shigeichi HIRASAWA

Almost Sure and Mean Convergence of Extended Stochastic Complexity

Masayuki GOTOH, Toshiyasu MATSUSHIMA, Shigeichi HIRASAWA

Full Text Views

0

Cite this

Summary :

We analyze the extended stochastic complexity (ESC) which has been proposed by K. Yamanishi. The ESC can be applied to learning algorithms for on-line prediction and batch-learning settings. Yamanishi derived the upper bound of ESC satisfying uniformly for all data sequences and that of the asymptotic expectation of ESC. However, Yamanishi concentrates mainly on the worst case performance and the lower bound has not been derived. In this paper, we show some interesting properties of ESC which are similar to Bayesian statistics: the Bayes rule and the asymptotic normality. We then derive the asymptotic formula of ESC in the meaning of almost sure and mean convergence within an error of o(1) using these properties.

Publication: IEICE TRANSACTIONS on Fundamentals Vol.E82-A No.10 pp.2129-2137

Publication Date: 1999/10/25

Publicized

Online ISSN

DOI

Type of Manuscript: Special Section PAPER (Special Section on Information Theory and Its Applications)

Category: Source Coding/Image Processing

Cite this

Copy

Masayuki GOTOH, Toshiyasu MATSUSHIMA, Shigeichi HIRASAWA, "Almost Sure and Mean Convergence of Extended Stochastic Complexity" in IEICE TRANSACTIONS on Fundamentals, vol. E82-A, no. 10, pp. 2129-2137, October 1999, doi: .
Abstract: We analyze the extended stochastic complexity (ESC) which has been proposed by K. Yamanishi. The ESC can be applied to learning algorithms for on-line prediction and batch-learning settings. Yamanishi derived the upper bound of ESC satisfying uniformly for all data sequences and that of the asymptotic expectation of ESC. However, Yamanishi concentrates mainly on the worst case performance and the lower bound has not been derived. In this paper, we show some interesting properties of ESC which are similar to Bayesian statistics: the Bayes rule and the asymptotic normality. We then derive the asymptotic formula of ESC in the meaning of almost sure and mean convergence within an error of o(1) using these properties.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/e82-a_10_2129/_p

Copy

@ARTICLE{e82-a_10_2129,
author={Masayuki GOTOH, Toshiyasu MATSUSHIMA, Shigeichi HIRASAWA, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Almost Sure and Mean Convergence of Extended Stochastic Complexity},
year={1999},
volume={E82-A},
number={10},
pages={2129-2137},
abstract={We analyze the extended stochastic complexity (ESC) which has been proposed by K. Yamanishi. The ESC can be applied to learning algorithms for on-line prediction and batch-learning settings. Yamanishi derived the upper bound of ESC satisfying uniformly for all data sequences and that of the asymptotic expectation of ESC. However, Yamanishi concentrates mainly on the worst case performance and the lower bound has not been derived. In this paper, we show some interesting properties of ESC which are similar to Bayesian statistics: the Bayes rule and the asymptotic normality. We then derive the asymptotic formula of ESC in the meaning of almost sure and mean convergence within an error of o(1) using these properties.},
keywords={},
doi={},
ISSN={},
month={October},}

Copy

TY - JOUR
TI - Almost Sure and Mean Convergence of Extended Stochastic Complexity
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 2129
EP - 2137
AU - Masayuki GOTOH
AU - Toshiyasu MATSUSHIMA
AU - Shigeichi HIRASAWA
PY - 1999
DO -
JO - IEICE TRANSACTIONS on Fundamentals
SN -
VL - E82-A
IS - 10
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - October 1999
AB - We analyze the extended stochastic complexity (ESC) which has been proposed by K. Yamanishi. The ESC can be applied to learning algorithms for on-line prediction and batch-learning settings. Yamanishi derived the upper bound of ESC satisfying uniformly for all data sequences and that of the asymptotic expectation of ESC. However, Yamanishi concentrates mainly on the worst case performance and the lower bound has not been derived. In this paper, we show some interesting properties of ESC which are similar to Bayesian statistics: the Bayes rule and the asymptotic normality. We then derive the asymptotic formula of ESC in the meaning of almost sure and mean convergence within an error of o(1) using these properties.
ER -