IEICE TRANSACTIONS on Fundamentals
Open Access
Capsule Network with Shortcut Routing
-
Full Text Views
66
- Cite this
- Free PDF (814.2KB)
Summary :
Capsules are fundamental informative units that are introduced into capsule networks to manipulate the hierarchical presentation of patterns. The part-hole relationship of an entity is learned through capsule layers, using a routing-by-agreement mechanism that is approximated by a voting procedure. Nevertheless, existing routing methods are computationally inefficient. We address this issue by proposing a novel routing mechanism, namely “shortcut routing”, that directly learns to activate global capsules from local capsules. In our method, the number of operations in the routing procedure is reduced by omitting the capsules in intermediate layers, resulting in lighter routing. To further address the computational problem, we investigate an attention-based approach, and propose fuzzy coefficients, which have been found to be efficient than mixture coefficients from EM routing. Our method achieves on-par classification results on the Mnist (99.52%), smallnorb (93.91%), and affNist (89.02%) datasets. Compared to EM routing, our fuzzy-based and attention-based routing methods attain reductions of 1.42 and 2.5 in terms of the number of calculations.
- Publication
- IEICE TRANSACTIONS on Fundamentals Vol.E104-A No.8 pp.1043-1050
- Publication Date
- 2021/08/01
- Publicized
- 2021/01/27
- Online ISSN
- 1745-1337
- DOI
- 10.1587/transfun.2020EAP1101
- Type of Manuscript
- PAPER
- Category
- Image
Authors
Thanh Vu DANG
Chonnam National University
Hoang Trong VO
Chonnam National University
Gwang Hyun YU
Chonnam National University
Jin Young KIM
Chonnam National University
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Thanh Vu DANG, Hoang Trong VO, Gwang Hyun YU, Jin Young KIM, "Capsule Network with Shortcut Routing" in IEICE TRANSACTIONS on Fundamentals,
vol. E104-A, no. 8, pp. 1043-1050, August 2021, doi: 10.1587/transfun.2020EAP1101.
Abstract: Capsules are fundamental informative units that are introduced into capsule networks to manipulate the hierarchical presentation of patterns. The part-hole relationship of an entity is learned through capsule layers, using a routing-by-agreement mechanism that is approximated by a voting procedure. Nevertheless, existing routing methods are computationally inefficient. We address this issue by proposing a novel routing mechanism, namely “shortcut routing”, that directly learns to activate global capsules from local capsules. In our method, the number of operations in the routing procedure is reduced by omitting the capsules in intermediate layers, resulting in lighter routing. To further address the computational problem, we investigate an attention-based approach, and propose fuzzy coefficients, which have been found to be efficient than mixture coefficients from EM routing. Our method achieves on-par classification results on the Mnist (99.52%), smallnorb (93.91%), and affNist (89.02%) datasets. Compared to EM routing, our fuzzy-based and attention-based routing methods attain reductions of 1.42 and 2.5 in terms of the number of calculations.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.2020EAP1101/_p
Copy
@ARTICLE{e104-a_8_1043,
author={Thanh Vu DANG, Hoang Trong VO, Gwang Hyun YU, Jin Young KIM, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Capsule Network with Shortcut Routing},
year={2021},
volume={E104-A},
number={8},
pages={1043-1050},
abstract={Capsules are fundamental informative units that are introduced into capsule networks to manipulate the hierarchical presentation of patterns. The part-hole relationship of an entity is learned through capsule layers, using a routing-by-agreement mechanism that is approximated by a voting procedure. Nevertheless, existing routing methods are computationally inefficient. We address this issue by proposing a novel routing mechanism, namely “shortcut routing”, that directly learns to activate global capsules from local capsules. In our method, the number of operations in the routing procedure is reduced by omitting the capsules in intermediate layers, resulting in lighter routing. To further address the computational problem, we investigate an attention-based approach, and propose fuzzy coefficients, which have been found to be efficient than mixture coefficients from EM routing. Our method achieves on-par classification results on the Mnist (99.52%), smallnorb (93.91%), and affNist (89.02%) datasets. Compared to EM routing, our fuzzy-based and attention-based routing methods attain reductions of 1.42 and 2.5 in terms of the number of calculations.},
keywords={},
doi={10.1587/transfun.2020EAP1101},
ISSN={1745-1337},
month={August},}
Copy
TY - JOUR
TI - Capsule Network with Shortcut Routing
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 1043
EP - 1050
AU - Thanh Vu DANG
AU - Hoang Trong VO
AU - Gwang Hyun YU
AU - Jin Young KIM
PY - 2021
DO - 10.1587/transfun.2020EAP1101
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E104-A
IS - 8
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - August 2021
AB - Capsules are fundamental informative units that are introduced into capsule networks to manipulate the hierarchical presentation of patterns. The part-hole relationship of an entity is learned through capsule layers, using a routing-by-agreement mechanism that is approximated by a voting procedure. Nevertheless, existing routing methods are computationally inefficient. We address this issue by proposing a novel routing mechanism, namely “shortcut routing”, that directly learns to activate global capsules from local capsules. In our method, the number of operations in the routing procedure is reduced by omitting the capsules in intermediate layers, resulting in lighter routing. To further address the computational problem, we investigate an attention-based approach, and propose fuzzy coefficients, which have been found to be efficient than mixture coefficients from EM routing. Our method achieves on-par classification results on the Mnist (99.52%), smallnorb (93.91%), and affNist (89.02%) datasets. Compared to EM routing, our fuzzy-based and attention-based routing methods attain reductions of 1.42 and 2.5 in terms of the number of calculations.
ER -
English
