IEICE TRANSACTIONS on Information
Penalized AdaBoost: Improving the Generalization Error of Gentle AdaBoost through a Margin Distribution
Summary :
Gentle AdaBoost is widely used in object detection and pattern recognition due to its efficiency and stability. To focus on instances with small margins, Gentle AdaBoost assigns larger weights to these instances during the training. However, misclassification of small-margin instances can still occur, which will cause the weights of these instances to become larger and larger. Eventually, several large-weight instances might dominate the whole data distribution, encouraging Gentle AdaBoost to choose weak hypotheses that fit only these instances in the late training phase. This phenomenon, known as “classifier distortion”, degrades the generalization error and can easily lead to overfitting since the deviation of all selected weak hypotheses is increased by the late-selected ones. To solve this problem, we propose a new variant which we call “Penalized AdaBoost”. In each iteration, our approach not only penalizes the misclassification of instances with small margins but also restrains the weight increase for instances with minimal margins. Our method performs better than Gentle AdaBoost because it avoids the “classifier distortion” effectively. Experiments show that our method achieves far lower generalization errors and a similar training speed compared with Gentle AdaBoost.
- Publication
- IEICE TRANSACTIONS on Information Vol.E98-D No.11 pp.1906-1915
- Publication Date
- 2015/11/01
- Publicized
- 2015/08/13
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.2015EDP7069
- Type of Manuscript
- PAPER
- Category
- Artificial Intelligence, Data Mining
Authors
Shuqiong WU
Tokyo Institute of Technology
Hiroshi NAGAHASHI
Tokyo Institute of Technology
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Shuqiong WU, Hiroshi NAGAHASHI, "Penalized AdaBoost: Improving the Generalization Error of Gentle AdaBoost through a Margin Distribution" in IEICE TRANSACTIONS on Information,
vol. E98-D, no. 11, pp. 1906-1915, November 2015, doi: 10.1587/transinf.2015EDP7069.
Abstract: Gentle AdaBoost is widely used in object detection and pattern recognition due to its efficiency and stability. To focus on instances with small margins, Gentle AdaBoost assigns larger weights to these instances during the training. However, misclassification of small-margin instances can still occur, which will cause the weights of these instances to become larger and larger. Eventually, several large-weight instances might dominate the whole data distribution, encouraging Gentle AdaBoost to choose weak hypotheses that fit only these instances in the late training phase. This phenomenon, known as “classifier distortion”, degrades the generalization error and can easily lead to overfitting since the deviation of all selected weak hypotheses is increased by the late-selected ones. To solve this problem, we propose a new variant which we call “Penalized AdaBoost”. In each iteration, our approach not only penalizes the misclassification of instances with small margins but also restrains the weight increase for instances with minimal margins. Our method performs better than Gentle AdaBoost because it avoids the “classifier distortion” effectively. Experiments show that our method achieves far lower generalization errors and a similar training speed compared with Gentle AdaBoost.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2015EDP7069/_p
Copy
@ARTICLE{e98-d_11_1906,
author={Shuqiong WU, Hiroshi NAGAHASHI, },
journal={IEICE TRANSACTIONS on Information},
title={Penalized AdaBoost: Improving the Generalization Error of Gentle AdaBoost through a Margin Distribution},
year={2015},
volume={E98-D},
number={11},
pages={1906-1915},
abstract={Gentle AdaBoost is widely used in object detection and pattern recognition due to its efficiency and stability. To focus on instances with small margins, Gentle AdaBoost assigns larger weights to these instances during the training. However, misclassification of small-margin instances can still occur, which will cause the weights of these instances to become larger and larger. Eventually, several large-weight instances might dominate the whole data distribution, encouraging Gentle AdaBoost to choose weak hypotheses that fit only these instances in the late training phase. This phenomenon, known as “classifier distortion”, degrades the generalization error and can easily lead to overfitting since the deviation of all selected weak hypotheses is increased by the late-selected ones. To solve this problem, we propose a new variant which we call “Penalized AdaBoost”. In each iteration, our approach not only penalizes the misclassification of instances with small margins but also restrains the weight increase for instances with minimal margins. Our method performs better than Gentle AdaBoost because it avoids the “classifier distortion” effectively. Experiments show that our method achieves far lower generalization errors and a similar training speed compared with Gentle AdaBoost.},
keywords={},
doi={10.1587/transinf.2015EDP7069},
ISSN={1745-1361},
month={November},}
Copy
TY - JOUR
TI - Penalized AdaBoost: Improving the Generalization Error of Gentle AdaBoost through a Margin Distribution
T2 - IEICE TRANSACTIONS on Information
SP - 1906
EP - 1915
AU - Shuqiong WU
AU - Hiroshi NAGAHASHI
PY - 2015
DO - 10.1587/transinf.2015EDP7069
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E98-D
IS - 11
JA - IEICE TRANSACTIONS on Information
Y1 - November 2015
AB - Gentle AdaBoost is widely used in object detection and pattern recognition due to its efficiency and stability. To focus on instances with small margins, Gentle AdaBoost assigns larger weights to these instances during the training. However, misclassification of small-margin instances can still occur, which will cause the weights of these instances to become larger and larger. Eventually, several large-weight instances might dominate the whole data distribution, encouraging Gentle AdaBoost to choose weak hypotheses that fit only these instances in the late training phase. This phenomenon, known as “classifier distortion”, degrades the generalization error and can easily lead to overfitting since the deviation of all selected weak hypotheses is increased by the late-selected ones. To solve this problem, we propose a new variant which we call “Penalized AdaBoost”. In each iteration, our approach not only penalizes the misclassification of instances with small margins but also restrains the weight increase for instances with minimal margins. Our method performs better than Gentle AdaBoost because it avoids the “classifier distortion” effectively. Experiments show that our method achieves far lower generalization errors and a similar training speed compared with Gentle AdaBoost.
ER -
English
