Improved Sample Complexity Bounds for Parameter Estimation
Summary :
Various authors have proposed probabilistic extensions of Valiant's PAC (Probably Approximately Correct) learning model in which the target to be learned is a (conditional) probability distribution. In this paper, we improve upon the best known upper bounds on the sample complexity of the parameter estimation part of the learning problem for distributions and stochastic rules over a finite domain with respect to the Kullback-Leibler divergence (KL-devergence). In particular, we improve the upper bound of order O(1/ε2) due to Abe, Takeuchi, and Warmuth to a bound of order O(1/ε). In obtaining our results, we made use of the properties of a specific estimator (slightly modified maximum likelihood estimator) with respect to the KL-divergence, while previously known upper bounds were obtained using the uniform convergence technique.
- Publication
- IEICE TRANSACTIONS on Information Vol.E78-D No.5 pp.526-531
- Publication Date
- 1995/05/25
- Publicized
- Online ISSN
- DOI
- Type of Manuscript
- Special Section PAPER (Special Issue on Algorithmic Learning Theory)
- Category
- Computational Learning Theory
Authors
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Jun-ichi TAKEUCHI, "Improved Sample Complexity Bounds for Parameter Estimation" in IEICE TRANSACTIONS on Information,
vol. E78-D, no. 5, pp. 526-531, May 1995, doi: .
Abstract: Various authors have proposed probabilistic extensions of Valiant's PAC (Probably Approximately Correct) learning model in which the target to be learned is a (conditional) probability distribution. In this paper, we improve upon the best known upper bounds on the sample complexity of the parameter estimation part of the learning problem for distributions and stochastic rules over a finite domain with respect to the Kullback-Leibler divergence (KL-devergence). In particular, we improve the upper bound of order O(1/ε2) due to Abe, Takeuchi, and Warmuth to a bound of order O(1/ε). In obtaining our results, we made use of the properties of a specific estimator (slightly modified maximum likelihood estimator) with respect to the KL-divergence, while previously known upper bounds were obtained using the uniform convergence technique.
URL: https://global.ieice.org/en_transactions/information/10.1587/e78-d_5_526/_p
Copy
@ARTICLE{e78-d_5_526,
author={Jun-ichi TAKEUCHI, },
journal={IEICE TRANSACTIONS on Information},
title={Improved Sample Complexity Bounds for Parameter Estimation},
year={1995},
volume={E78-D},
number={5},
pages={526-531},
abstract={Various authors have proposed probabilistic extensions of Valiant's PAC (Probably Approximately Correct) learning model in which the target to be learned is a (conditional) probability distribution. In this paper, we improve upon the best known upper bounds on the sample complexity of the parameter estimation part of the learning problem for distributions and stochastic rules over a finite domain with respect to the Kullback-Leibler divergence (KL-devergence). In particular, we improve the upper bound of order O(1/ε2) due to Abe, Takeuchi, and Warmuth to a bound of order O(1/ε). In obtaining our results, we made use of the properties of a specific estimator (slightly modified maximum likelihood estimator) with respect to the KL-divergence, while previously known upper bounds were obtained using the uniform convergence technique.},
keywords={},
doi={},
ISSN={},
month={May},}
Copy
TY - JOUR
TI - Improved Sample Complexity Bounds for Parameter Estimation
T2 - IEICE TRANSACTIONS on Information
SP - 526
EP - 531
AU - Jun-ichi TAKEUCHI
PY - 1995
DO -
JO - IEICE TRANSACTIONS on Information
SN -
VL - E78-D
IS - 5
JA - IEICE TRANSACTIONS on Information
Y1 - May 1995
AB - Various authors have proposed probabilistic extensions of Valiant's PAC (Probably Approximately Correct) learning model in which the target to be learned is a (conditional) probability distribution. In this paper, we improve upon the best known upper bounds on the sample complexity of the parameter estimation part of the learning problem for distributions and stochastic rules over a finite domain with respect to the Kullback-Leibler divergence (KL-devergence). In particular, we improve the upper bound of order O(1/ε2) due to Abe, Takeuchi, and Warmuth to a bound of order O(1/ε). In obtaining our results, we made use of the properties of a specific estimator (slightly modified maximum likelihood estimator) with respect to the KL-divergence, while previously known upper bounds were obtained using the uniform convergence technique.
ER -
English
