IEICE TRANSACTIONS on Fundamentals
Efficient Secure Neural Network Prediction Protocol Reducing Accuracy Degradation
Summary :
Machine learning models inherently memorize significant amounts of information, and thus hiding not only prediction processes but also trained models, i.e., model obliviousness, is desirable in the cloud setting. Several works achieved model obliviousness with the MNIST dataset, but datasets that include complicated samples, e.g., CIFAR-10 and CIFAR-100, are also used in actual applications, such as face recognition. Secret sharing-based secure prediction for CIFAR-10 is difficult to achieve. When a deep layer architecture such as CNN is used, the calculation error when performing secret calculation becomes large and the accuracy deteriorates. In addition, if detailed calculations are performed to improve accuracy, a large amount of calculation is required. Therefore, even if the conventional method is applied to CNN as it is, good results as described in the paper cannot be obtained. In this paper, we propose two approaches to solve this problem. Firstly, we propose a new protocol named Batch-normalizedActivation that combines BatchNormalization and Activation. Since BatchNormalization includes real number operations, when performing secret calculation, parameters must be converted into integers, which causes a calculation error and decrease accuracy. By using our protocol, calculation errors can be eliminated, and accuracy degradation can be eliminated. Further, the processing is simplified, and the amount of calculation is reduced. Secondly, we explore a secret computation friendly and high accuracy architecture. Related works use a low-accuracy, simple architecture, but in reality, a high accuracy architecture should be used. Therefore, we also explored a high accuracy architecture for the CIFAR10 dataset. Our proposed protocol can compute prediction of CIFAR-10 within 15.05 seconds with 87.36% accuracy while providing model obliviousness.
- Publication
- IEICE TRANSACTIONS on Fundamentals Vol.E103-A No.12 pp.1367-1380
- Publication Date
- 2020/12/01
- Publicized
- Online ISSN
- 1745-1337
- DOI
- 10.1587/transfun.2020TAP0011
- Type of Manuscript
- Special Section PAPER (Special Section on Information Theory and Its Applications)
- Category
- Cryptography and Information Security
Authors
Naohisa NISHIDA
Panasonic Corporation
Tatsumi OBA
Panasonic Corporation
Yuji UNAGAMI
Panasonic Corporation
Jason PAUL CRUZ
Osaka University
Naoto YANAI
Osaka University
Tadanori TERUYA
National Institute of Advanced Industrial Science and Technology
Nuttapong ATTRAPADUNG
National Institute of Advanced Industrial Science and Technology
Takahiro MATSUDA
National Institute of Advanced Industrial Science and Technology
Goichiro HANAOKA
National Institute of Advanced Industrial Science and Technology
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Naohisa NISHIDA, Tatsumi OBA, Yuji UNAGAMI, Jason PAUL CRUZ, Naoto YANAI, Tadanori TERUYA, Nuttapong ATTRAPADUNG, Takahiro MATSUDA, Goichiro HANAOKA, "Efficient Secure Neural Network Prediction Protocol Reducing Accuracy Degradation" in IEICE TRANSACTIONS on Fundamentals,
vol. E103-A, no. 12, pp. 1367-1380, December 2020, doi: 10.1587/transfun.2020TAP0011.
Abstract: Machine learning models inherently memorize significant amounts of information, and thus hiding not only prediction processes but also trained models, i.e., model obliviousness, is desirable in the cloud setting. Several works achieved model obliviousness with the MNIST dataset, but datasets that include complicated samples, e.g., CIFAR-10 and CIFAR-100, are also used in actual applications, such as face recognition. Secret sharing-based secure prediction for CIFAR-10 is difficult to achieve. When a deep layer architecture such as CNN is used, the calculation error when performing secret calculation becomes large and the accuracy deteriorates. In addition, if detailed calculations are performed to improve accuracy, a large amount of calculation is required. Therefore, even if the conventional method is applied to CNN as it is, good results as described in the paper cannot be obtained. In this paper, we propose two approaches to solve this problem. Firstly, we propose a new protocol named Batch-normalizedActivation that combines BatchNormalization and Activation. Since BatchNormalization includes real number operations, when performing secret calculation, parameters must be converted into integers, which causes a calculation error and decrease accuracy. By using our protocol, calculation errors can be eliminated, and accuracy degradation can be eliminated. Further, the processing is simplified, and the amount of calculation is reduced. Secondly, we explore a secret computation friendly and high accuracy architecture. Related works use a low-accuracy, simple architecture, but in reality, a high accuracy architecture should be used. Therefore, we also explored a high accuracy architecture for the CIFAR10 dataset. Our proposed protocol can compute prediction of CIFAR-10 within 15.05 seconds with 87.36% accuracy while providing model obliviousness.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.2020TAP0011/_p
Copy
@ARTICLE{e103-a_12_1367,
author={Naohisa NISHIDA, Tatsumi OBA, Yuji UNAGAMI, Jason PAUL CRUZ, Naoto YANAI, Tadanori TERUYA, Nuttapong ATTRAPADUNG, Takahiro MATSUDA, Goichiro HANAOKA, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Efficient Secure Neural Network Prediction Protocol Reducing Accuracy Degradation},
year={2020},
volume={E103-A},
number={12},
pages={1367-1380},
abstract={Machine learning models inherently memorize significant amounts of information, and thus hiding not only prediction processes but also trained models, i.e., model obliviousness, is desirable in the cloud setting. Several works achieved model obliviousness with the MNIST dataset, but datasets that include complicated samples, e.g., CIFAR-10 and CIFAR-100, are also used in actual applications, such as face recognition. Secret sharing-based secure prediction for CIFAR-10 is difficult to achieve. When a deep layer architecture such as CNN is used, the calculation error when performing secret calculation becomes large and the accuracy deteriorates. In addition, if detailed calculations are performed to improve accuracy, a large amount of calculation is required. Therefore, even if the conventional method is applied to CNN as it is, good results as described in the paper cannot be obtained. In this paper, we propose two approaches to solve this problem. Firstly, we propose a new protocol named Batch-normalizedActivation that combines BatchNormalization and Activation. Since BatchNormalization includes real number operations, when performing secret calculation, parameters must be converted into integers, which causes a calculation error and decrease accuracy. By using our protocol, calculation errors can be eliminated, and accuracy degradation can be eliminated. Further, the processing is simplified, and the amount of calculation is reduced. Secondly, we explore a secret computation friendly and high accuracy architecture. Related works use a low-accuracy, simple architecture, but in reality, a high accuracy architecture should be used. Therefore, we also explored a high accuracy architecture for the CIFAR10 dataset. Our proposed protocol can compute prediction of CIFAR-10 within 15.05 seconds with 87.36% accuracy while providing model obliviousness.},
keywords={},
doi={10.1587/transfun.2020TAP0011},
ISSN={1745-1337},
month={December},}
Copy
TY - JOUR
TI - Efficient Secure Neural Network Prediction Protocol Reducing Accuracy Degradation
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 1367
EP - 1380
AU - Naohisa NISHIDA
AU - Tatsumi OBA
AU - Yuji UNAGAMI
AU - Jason PAUL CRUZ
AU - Naoto YANAI
AU - Tadanori TERUYA
AU - Nuttapong ATTRAPADUNG
AU - Takahiro MATSUDA
AU - Goichiro HANAOKA
PY - 2020
DO - 10.1587/transfun.2020TAP0011
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E103-A
IS - 12
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - December 2020
AB - Machine learning models inherently memorize significant amounts of information, and thus hiding not only prediction processes but also trained models, i.e., model obliviousness, is desirable in the cloud setting. Several works achieved model obliviousness with the MNIST dataset, but datasets that include complicated samples, e.g., CIFAR-10 and CIFAR-100, are also used in actual applications, such as face recognition. Secret sharing-based secure prediction for CIFAR-10 is difficult to achieve. When a deep layer architecture such as CNN is used, the calculation error when performing secret calculation becomes large and the accuracy deteriorates. In addition, if detailed calculations are performed to improve accuracy, a large amount of calculation is required. Therefore, even if the conventional method is applied to CNN as it is, good results as described in the paper cannot be obtained. In this paper, we propose two approaches to solve this problem. Firstly, we propose a new protocol named Batch-normalizedActivation that combines BatchNormalization and Activation. Since BatchNormalization includes real number operations, when performing secret calculation, parameters must be converted into integers, which causes a calculation error and decrease accuracy. By using our protocol, calculation errors can be eliminated, and accuracy degradation can be eliminated. Further, the processing is simplified, and the amount of calculation is reduced. Secondly, we explore a secret computation friendly and high accuracy architecture. Related works use a low-accuracy, simple architecture, but in reality, a high accuracy architecture should be used. Therefore, we also explored a high accuracy architecture for the CIFAR10 dataset. Our proposed protocol can compute prediction of CIFAR-10 within 15.05 seconds with 87.36% accuracy while providing model obliviousness.
ER -
English
