IEICE TRANSACTIONS on Information
CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy
Summary :
In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.
- Publication
- IEICE TRANSACTIONS on Information Vol.E106-D No.4 pp.523-537
- Publication Date
- 2023/04/01
- Publicized
- 2023/01/23
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.2022EDP7200
- Type of Manuscript
- PAPER
- Category
- Artificial Intelligence, Data Mining
Authors
Daiki NISHIYAMA
University of Tsukuba,RIKEN Center for Advanced Intelligence Project
Kazuto FUKUCHI
RIKEN Center for Advanced Intelligence Project,University of Tsukuba
Youhei AKIMOTO
RIKEN Center for Advanced Intelligence Project,University of Tsukuba
Jun SAKUMA
RIKEN Center for Advanced Intelligence Project,University of Tsukuba
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Daiki NISHIYAMA, Kazuto FUKUCHI, Youhei AKIMOTO, Jun SAKUMA, "CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy" in IEICE TRANSACTIONS on Information,
vol. E106-D, no. 4, pp. 523-537, April 2023, doi: 10.1587/transinf.2022EDP7200.
Abstract: In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2022EDP7200/_p
Copy
@ARTICLE{e106-d_4_523,
author={Daiki NISHIYAMA, Kazuto FUKUCHI, Youhei AKIMOTO, Jun SAKUMA, },
journal={IEICE TRANSACTIONS on Information},
title={CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy},
year={2023},
volume={E106-D},
number={4},
pages={523-537},
abstract={In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.},
keywords={},
doi={10.1587/transinf.2022EDP7200},
ISSN={1745-1361},
month={April},}
Copy
TY - JOUR
TI - CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy
T2 - IEICE TRANSACTIONS on Information
SP - 523
EP - 537
AU - Daiki NISHIYAMA
AU - Kazuto FUKUCHI
AU - Youhei AKIMOTO
AU - Jun SAKUMA
PY - 2023
DO - 10.1587/transinf.2022EDP7200
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E106-D
IS - 4
JA - IEICE TRANSACTIONS on Information
Y1 - April 2023
AB - In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.
ER -
English
