IEICE TRANSACTIONS on Information
Stochastic Discrete First-Order Algorithm for Feature Subset Selection
Summary :
This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.
- Publication
- IEICE TRANSACTIONS on Information Vol.E103-D No.7 pp.1693-1702
- Publication Date
- 2020/07/01
- Publicized
- 2020/04/13
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.2019EDP7274
- Type of Manuscript
- PAPER
- Category
- Artificial Intelligence, Data Mining
Authors
Kota KUDO
University of Tsukuba
Yuichi TAKANO
University of Tsukuba
Ryo NOMURA
Waseda University
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Kota KUDO, Yuichi TAKANO, Ryo NOMURA, "Stochastic Discrete First-Order Algorithm for Feature Subset Selection" in IEICE TRANSACTIONS on Information,
vol. E103-D, no. 7, pp. 1693-1702, July 2020, doi: 10.1587/transinf.2019EDP7274.
Abstract: This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019EDP7274/_p
Copy
@ARTICLE{e103-d_7_1693,
author={Kota KUDO, Yuichi TAKANO, Ryo NOMURA, },
journal={IEICE TRANSACTIONS on Information},
title={Stochastic Discrete First-Order Algorithm for Feature Subset Selection},
year={2020},
volume={E103-D},
number={7},
pages={1693-1702},
abstract={This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.},
keywords={},
doi={10.1587/transinf.2019EDP7274},
ISSN={1745-1361},
month={July},}
Copy
TY - JOUR
TI - Stochastic Discrete First-Order Algorithm for Feature Subset Selection
T2 - IEICE TRANSACTIONS on Information
SP - 1693
EP - 1702
AU - Kota KUDO
AU - Yuichi TAKANO
AU - Ryo NOMURA
PY - 2020
DO - 10.1587/transinf.2019EDP7274
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E103-D
IS - 7
JA - IEICE TRANSACTIONS on Information
Y1 - July 2020
AB - This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.
ER -
English
