IEICE TRANSACTIONS on Information
A Spectral Clustering Based Filter-Level Pruning Method for Convolutional Neural Networks
Summary :
Convolutional Neural Networks (CNNs) usually have millions or even billions of parameters, which make them hard to be deployed into mobile devices. In this work, we present a novel filter-level pruning method to alleviate this issue. More concretely, we first construct an undirected fully connected graph to represent a pre-trained CNN model. Then, we employ the spectral clustering algorithm to divide the graph into some subgraphs, which is equivalent to clustering the similar filters of the CNN into the same groups. After gaining the grouping relationships among the filters, we finally keep one filter for one group and retrain the pruned model. Compared with previous pruning methods that identify the redundant filters by heuristic ways, the proposed method can select the pruning candidates more reasonably and precisely. Experimental results also show that our proposed pruning method has significant improvements over the state-of-the-arts.
- Publication
- IEICE TRANSACTIONS on Information Vol.E102-D No.12 pp.2624-2627
- Publication Date
- 2019/12/01
- Publicized
- 2019/09/17
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.2019EDL8118
- Type of Manuscript
- LETTER
- Category
- Artificial Intelligence, Data Mining
Authors
Lianqiang LI
Shanghai Jiao Tong University (SJTU)
Jie ZHU
Shanghai Jiao Tong University (SJTU)
Ming-Ting SUN
University of Washington
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Lianqiang LI, Jie ZHU, Ming-Ting SUN, "A Spectral Clustering Based Filter-Level Pruning Method for Convolutional Neural Networks" in IEICE TRANSACTIONS on Information,
vol. E102-D, no. 12, pp. 2624-2627, December 2019, doi: 10.1587/transinf.2019EDL8118.
Abstract: Convolutional Neural Networks (CNNs) usually have millions or even billions of parameters, which make them hard to be deployed into mobile devices. In this work, we present a novel filter-level pruning method to alleviate this issue. More concretely, we first construct an undirected fully connected graph to represent a pre-trained CNN model. Then, we employ the spectral clustering algorithm to divide the graph into some subgraphs, which is equivalent to clustering the similar filters of the CNN into the same groups. After gaining the grouping relationships among the filters, we finally keep one filter for one group and retrain the pruned model. Compared with previous pruning methods that identify the redundant filters by heuristic ways, the proposed method can select the pruning candidates more reasonably and precisely. Experimental results also show that our proposed pruning method has significant improvements over the state-of-the-arts.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019EDL8118/_p
Copy
@ARTICLE{e102-d_12_2624,
author={Lianqiang LI, Jie ZHU, Ming-Ting SUN, },
journal={IEICE TRANSACTIONS on Information},
title={A Spectral Clustering Based Filter-Level Pruning Method for Convolutional Neural Networks},
year={2019},
volume={E102-D},
number={12},
pages={2624-2627},
abstract={Convolutional Neural Networks (CNNs) usually have millions or even billions of parameters, which make them hard to be deployed into mobile devices. In this work, we present a novel filter-level pruning method to alleviate this issue. More concretely, we first construct an undirected fully connected graph to represent a pre-trained CNN model. Then, we employ the spectral clustering algorithm to divide the graph into some subgraphs, which is equivalent to clustering the similar filters of the CNN into the same groups. After gaining the grouping relationships among the filters, we finally keep one filter for one group and retrain the pruned model. Compared with previous pruning methods that identify the redundant filters by heuristic ways, the proposed method can select the pruning candidates more reasonably and precisely. Experimental results also show that our proposed pruning method has significant improvements over the state-of-the-arts.},
keywords={},
doi={10.1587/transinf.2019EDL8118},
ISSN={1745-1361},
month={December},}
Copy
TY - JOUR
TI - A Spectral Clustering Based Filter-Level Pruning Method for Convolutional Neural Networks
T2 - IEICE TRANSACTIONS on Information
SP - 2624
EP - 2627
AU - Lianqiang LI
AU - Jie ZHU
AU - Ming-Ting SUN
PY - 2019
DO - 10.1587/transinf.2019EDL8118
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E102-D
IS - 12
JA - IEICE TRANSACTIONS on Information
Y1 - December 2019
AB - Convolutional Neural Networks (CNNs) usually have millions or even billions of parameters, which make them hard to be deployed into mobile devices. In this work, we present a novel filter-level pruning method to alleviate this issue. More concretely, we first construct an undirected fully connected graph to represent a pre-trained CNN model. Then, we employ the spectral clustering algorithm to divide the graph into some subgraphs, which is equivalent to clustering the similar filters of the CNN into the same groups. After gaining the grouping relationships among the filters, we finally keep one filter for one group and retrain the pruned model. Compared with previous pruning methods that identify the redundant filters by heuristic ways, the proposed method can select the pruning candidates more reasonably and precisely. Experimental results also show that our proposed pruning method has significant improvements over the state-of-the-arts.
ER -
English
