The search functionality is under construction.

The search functionality is under construction.

This paper presents a pruning method, Reconstruction Error Aware Pruning (REAP), to reduce the redundancy of convolutional neural network models for accelerating their inference. In REAP, we have the following steps: 1) Prune the channels whose outputs are redundant and can be reconstructed from the outputs of other channels in each convolutional layer; 2) Update the weights of the remaining channels by least squares method so as to compensate the error caused by pruning. This is how we compress and accelerate the models that are initially large and slow with little degradation. The ability of REAP to maintain the model performances saves us lots of time and labors for retraining the pruned models. The challenge of REAP is the computational cost for selecting the channels to be pruned. For selecting the channels, we need to solve a huge number of least squares problems. We have developed an efficient algorithm based on biorthogonal system to obtain the solutions of those least squares problems. In the experiments, we show that REAP can conduct pruning with smaller sacrifice of the model performances than several existing methods including the previously state-of-the-art one.

- Publication
- IEICE TRANSACTIONS on Information Vol.E104-D No.1 pp.194-202

- Publication Date
- 2021/01/01

- Publicized
- 2020/10/02

- Online ISSN
- 1745-1361

- DOI
- 10.1587/transinf.2020EDP7049

- Type of Manuscript
- PAPER

- Category
- Biocybernetics, Neurocomputing

Koji KAMMA

Wakayama University

Toshikazu WADA

Wakayama University

The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.

Copy

Koji KAMMA, Toshikazu WADA, "REAP: A Method for Pruning Convolutional Neural Networks with Performance Preservation" in IEICE TRANSACTIONS on Information,
vol. E104-D, no. 1, pp. 194-202, January 2021, doi: 10.1587/transinf.2020EDP7049.

Abstract: This paper presents a pruning method, Reconstruction Error Aware Pruning (REAP), to reduce the redundancy of convolutional neural network models for accelerating their inference. In REAP, we have the following steps: 1) Prune the channels whose outputs are redundant and can be reconstructed from the outputs of other channels in each convolutional layer; 2) Update the weights of the remaining channels by least squares method so as to compensate the error caused by pruning. This is how we compress and accelerate the models that are initially large and slow with little degradation. The ability of REAP to maintain the model performances saves us lots of time and labors for retraining the pruned models. The challenge of REAP is the computational cost for selecting the channels to be pruned. For selecting the channels, we need to solve a huge number of least squares problems. We have developed an efficient algorithm based on biorthogonal system to obtain the solutions of those least squares problems. In the experiments, we show that REAP can conduct pruning with smaller sacrifice of the model performances than several existing methods including the previously state-of-the-art one.

URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2020EDP7049/_p

Copy

@ARTICLE{e104-d_1_194,

author={Koji KAMMA, Toshikazu WADA, },

journal={IEICE TRANSACTIONS on Information},

title={REAP: A Method for Pruning Convolutional Neural Networks with Performance Preservation},

year={2021},

volume={E104-D},

number={1},

pages={194-202},

abstract={This paper presents a pruning method, Reconstruction Error Aware Pruning (REAP), to reduce the redundancy of convolutional neural network models for accelerating their inference. In REAP, we have the following steps: 1) Prune the channels whose outputs are redundant and can be reconstructed from the outputs of other channels in each convolutional layer; 2) Update the weights of the remaining channels by least squares method so as to compensate the error caused by pruning. This is how we compress and accelerate the models that are initially large and slow with little degradation. The ability of REAP to maintain the model performances saves us lots of time and labors for retraining the pruned models. The challenge of REAP is the computational cost for selecting the channels to be pruned. For selecting the channels, we need to solve a huge number of least squares problems. We have developed an efficient algorithm based on biorthogonal system to obtain the solutions of those least squares problems. In the experiments, we show that REAP can conduct pruning with smaller sacrifice of the model performances than several existing methods including the previously state-of-the-art one.},

keywords={},

doi={10.1587/transinf.2020EDP7049},

ISSN={1745-1361},

month={January},}

Copy

TY - JOUR

TI - REAP: A Method for Pruning Convolutional Neural Networks with Performance Preservation

T2 - IEICE TRANSACTIONS on Information

SP - 194

EP - 202

AU - Koji KAMMA

AU - Toshikazu WADA

PY - 2021

DO - 10.1587/transinf.2020EDP7049

JO - IEICE TRANSACTIONS on Information

SN - 1745-1361

VL - E104-D

IS - 1

JA - IEICE TRANSACTIONS on Information

Y1 - January 2021

AB - This paper presents a pruning method, Reconstruction Error Aware Pruning (REAP), to reduce the redundancy of convolutional neural network models for accelerating their inference. In REAP, we have the following steps: 1) Prune the channels whose outputs are redundant and can be reconstructed from the outputs of other channels in each convolutional layer; 2) Update the weights of the remaining channels by least squares method so as to compensate the error caused by pruning. This is how we compress and accelerate the models that are initially large and slow with little degradation. The ability of REAP to maintain the model performances saves us lots of time and labors for retraining the pruned models. The challenge of REAP is the computational cost for selecting the channels to be pruned. For selecting the channels, we need to solve a huge number of least squares problems. We have developed an efficient algorithm based on biorthogonal system to obtain the solutions of those least squares problems. In the experiments, we show that REAP can conduct pruning with smaller sacrifice of the model performances than several existing methods including the previously state-of-the-art one.

ER -