CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy

Daiki NISHIYAMA; Kazuto FUKUCHI; Youhei AKIMOTO; Jun SAKUMA

doi:10.1587/transinf.2022EDP7200

CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy

Daiki NISHIYAMA, Kazuto FUKUCHI, Youhei AKIMOTO, Jun SAKUMA

Full Text Views

7

Cite this

Summary :

In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.

Publication: IEICE TRANSACTIONS on Information Vol.E106-D No.4 pp.523-537

Publication Date: 2023/04/01

Publicized: 2023/01/23

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2022EDP7200

Type of Manuscript: PAPER

Category: Artificial Intelligence, Data Mining

Authors

Daiki NISHIYAMA
  University of Tsukuba,RIKEN Center for Advanced Intelligence Project
Kazuto FUKUCHI
  RIKEN Center for Advanced Intelligence Project,University of Tsukuba
Youhei AKIMOTO
  RIKEN Center for Advanced Intelligence Project,University of Tsukuba
Jun SAKUMA
  RIKEN Center for Advanced Intelligence Project,University of Tsukuba

Keyword

machine learning, multiclass classification, loss function, neural networks, cost-sensitive learning, deep learning

Cite this

Copy

Daiki NISHIYAMA, Kazuto FUKUCHI, Youhei AKIMOTO, Jun SAKUMA, "CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy" in IEICE TRANSACTIONS on Information, vol. E106-D, no. 4, pp. 523-537, April 2023, doi: 10.1587/transinf.2022EDP7200.
Abstract: In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2022EDP7200/_p

Copy

@ARTICLE{e106-d_4_523,
author={Daiki NISHIYAMA, Kazuto FUKUCHI, Youhei AKIMOTO, Jun SAKUMA, },
journal={IEICE TRANSACTIONS on Information},
title={CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy},
year={2023},
volume={E106-D},
number={4},
pages={523-537},
abstract={In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.},
keywords={},
doi={10.1587/transinf.2022EDP7200},
ISSN={1745-1361},
month={April},}

Copy

TY - JOUR
TI - CAMRI Loss: Improving the Recall of a Specific Class without Sacrificing Accuracy
T2 - IEICE TRANSACTIONS on Information
SP - 523
EP - 537
AU - Daiki NISHIYAMA
AU - Kazuto FUKUCHI
AU - Youhei AKIMOTO
AU - Jun SAKUMA
PY - 2023
DO - 10.1587/transinf.2022EDP7200
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E106-D
IS - 4
JA - IEICE TRANSACTIONS on Information
Y1 - April 2023
AB - In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.
ER -