A Learning Algorithm of Boosting Kernel Discriminant Analysis for Pattern Recognition

Shinji KITA; Seiichi OZAWA; Satoshi MAEKAWA; Shigeo ABE

doi:10.1093/ietisy/e90-d.11.1853

A Learning Algorithm of Boosting Kernel Discriminant Analysis for Pattern Recognition

Shinji KITA, Seiichi OZAWA, Satoshi MAEKAWA, Shigeo ABE

Full Text Views

0

Cite this

Summary :

In this paper, we present a new method to enhance classification performance of a multiple classifier system by combining a boosting technique called AdaBoost.M2 and Kernel Discriminant Analysis (KDA). To reduce the dependency between classifier outputs and to speed up the learning, each classifier is trained in a different feature space, which is obtained by applying KDA to a small set of hard-to-classify training samples. The training of the system is conducted based on AdaBoost.M2, and the classifiers are implemented by Radial Basis Function networks. To perform KDA at every boosting round in a realistic time scale, a new kernel selection method based on the class separability measure is proposed. Furthermore, a new criterion of the training convergence is also proposed to acquire good classification performance with fewer boosting rounds. To evaluate the proposed method, several experiments are carried out using standard evaluation datasets. The experimental results demonstrate that the proposed method can select an optimal kernel parameter more efficiently than the conventional cross-validation method, and that the training of boosting classifiers is terminated with a fairly small number of rounds to attain good classification accuracy. For multi-class classification problems, the proposed method outperforms both Boosting Linear Discriminant Analysis (BLDA) and Radial-Basis Function Network (RBFN) with regard to the classification accuracy. On the other hand, the performance evaluation for 2-class problems shows that the advantage of the proposed BKDA against BLDA and RBFN depends on the datasets.

Publication: IEICE TRANSACTIONS on Information Vol.E90-D No.11 pp.1853-1863

Publication Date: 2007/11/01

Publicized

Online ISSN: 1745-1361

DOI: 10.1093/ietisy/e90-d.11.1853

Type of Manuscript: PAPER

Category: Biocybernetics, Neurocomputing

Cite this

Copy

Shinji KITA, Seiichi OZAWA, Satoshi MAEKAWA, Shigeo ABE, "A Learning Algorithm of Boosting Kernel Discriminant Analysis for Pattern Recognition" in IEICE TRANSACTIONS on Information, vol. E90-D, no. 11, pp. 1853-1863, November 2007, doi: 10.1093/ietisy/e90-d.11.1853.
Abstract: In this paper, we present a new method to enhance classification performance of a multiple classifier system by combining a boosting technique called AdaBoost.M2 and Kernel Discriminant Analysis (KDA). To reduce the dependency between classifier outputs and to speed up the learning, each classifier is trained in a different feature space, which is obtained by applying KDA to a small set of hard-to-classify training samples. The training of the system is conducted based on AdaBoost.M2, and the classifiers are implemented by Radial Basis Function networks. To perform KDA at every boosting round in a realistic time scale, a new kernel selection method based on the class separability measure is proposed. Furthermore, a new criterion of the training convergence is also proposed to acquire good classification performance with fewer boosting rounds. To evaluate the proposed method, several experiments are carried out using standard evaluation datasets. The experimental results demonstrate that the proposed method can select an optimal kernel parameter more efficiently than the conventional cross-validation method, and that the training of boosting classifiers is terminated with a fairly small number of rounds to attain good classification accuracy. For multi-class classification problems, the proposed method outperforms both Boosting Linear Discriminant Analysis (BLDA) and Radial-Basis Function Network (RBFN) with regard to the classification accuracy. On the other hand, the performance evaluation for 2-class problems shows that the advantage of the proposed BKDA against BLDA and RBFN depends on the datasets.
URL: https://global.ieice.org/en_transactions/information/10.1093/ietisy/e90-d.11.1853/_p

Copy

@ARTICLE{e90-d_11_1853,
author={Shinji KITA, Seiichi OZAWA, Satoshi MAEKAWA, Shigeo ABE, },
journal={IEICE TRANSACTIONS on Information},
title={A Learning Algorithm of Boosting Kernel Discriminant Analysis for Pattern Recognition},
year={2007},
volume={E90-D},
number={11},
pages={1853-1863},
abstract={In this paper, we present a new method to enhance classification performance of a multiple classifier system by combining a boosting technique called AdaBoost.M2 and Kernel Discriminant Analysis (KDA). To reduce the dependency between classifier outputs and to speed up the learning, each classifier is trained in a different feature space, which is obtained by applying KDA to a small set of hard-to-classify training samples. The training of the system is conducted based on AdaBoost.M2, and the classifiers are implemented by Radial Basis Function networks. To perform KDA at every boosting round in a realistic time scale, a new kernel selection method based on the class separability measure is proposed. Furthermore, a new criterion of the training convergence is also proposed to acquire good classification performance with fewer boosting rounds. To evaluate the proposed method, several experiments are carried out using standard evaluation datasets. The experimental results demonstrate that the proposed method can select an optimal kernel parameter more efficiently than the conventional cross-validation method, and that the training of boosting classifiers is terminated with a fairly small number of rounds to attain good classification accuracy. For multi-class classification problems, the proposed method outperforms both Boosting Linear Discriminant Analysis (BLDA) and Radial-Basis Function Network (RBFN) with regard to the classification accuracy. On the other hand, the performance evaluation for 2-class problems shows that the advantage of the proposed BKDA against BLDA and RBFN depends on the datasets.},
keywords={},
doi={10.1093/ietisy/e90-d.11.1853},
ISSN={1745-1361},
month={November},}

Copy

TY - JOUR
TI - A Learning Algorithm of Boosting Kernel Discriminant Analysis for Pattern Recognition
T2 - IEICE TRANSACTIONS on Information
SP - 1853
EP - 1863
AU - Shinji KITA
AU - Seiichi OZAWA
AU - Satoshi MAEKAWA
AU - Shigeo ABE
PY - 2007
DO - 10.1093/ietisy/e90-d.11.1853
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E90-D
IS - 11
JA - IEICE TRANSACTIONS on Information
Y1 - November 2007
AB - In this paper, we present a new method to enhance classification performance of a multiple classifier system by combining a boosting technique called AdaBoost.M2 and Kernel Discriminant Analysis (KDA). To reduce the dependency between classifier outputs and to speed up the learning, each classifier is trained in a different feature space, which is obtained by applying KDA to a small set of hard-to-classify training samples. The training of the system is conducted based on AdaBoost.M2, and the classifiers are implemented by Radial Basis Function networks. To perform KDA at every boosting round in a realistic time scale, a new kernel selection method based on the class separability measure is proposed. Furthermore, a new criterion of the training convergence is also proposed to acquire good classification performance with fewer boosting rounds. To evaluate the proposed method, several experiments are carried out using standard evaluation datasets. The experimental results demonstrate that the proposed method can select an optimal kernel parameter more efficiently than the conventional cross-validation method, and that the training of boosting classifiers is terminated with a fairly small number of rounds to attain good classification accuracy. For multi-class classification problems, the proposed method outperforms both Boosting Linear Discriminant Analysis (BLDA) and Radial-Basis Function Network (RBFN) with regard to the classification accuracy. On the other hand, the performance evaluation for 2-class problems shows that the advantage of the proposed BKDA against BLDA and RBFN depends on the datasets.
ER -