The search functionality is under construction.

The search functionality is under construction.

This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas *et al.* [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the *L*_{2}-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.

- Publication
- IEICE TRANSACTIONS on Information Vol.E103-D No.7 pp.1693-1702

- Publication Date
- 2020/07/01

- Publicized
- 2020/04/13

- Online ISSN
- 1745-1361

- DOI
- 10.1587/transinf.2019EDP7274

- Type of Manuscript
- PAPER

- Category
- Artificial Intelligence, Data Mining

Kota KUDO

University of Tsukuba

Yuichi TAKANO

University of Tsukuba

Ryo NOMURA

Waseda University

The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.

Copy

Kota KUDO, Yuichi TAKANO, Ryo NOMURA, "Stochastic Discrete First-Order Algorithm for Feature Subset Selection" in IEICE TRANSACTIONS on Information,
vol. E103-D, no. 7, pp. 1693-1702, July 2020, doi: 10.1587/transinf.2019EDP7274.

Abstract: This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas *et al.* [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the *L*_{2}-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.

URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019EDP7274/_p

Copy

@ARTICLE{e103-d_7_1693,

author={Kota KUDO, Yuichi TAKANO, Ryo NOMURA, },

journal={IEICE TRANSACTIONS on Information},

title={Stochastic Discrete First-Order Algorithm for Feature Subset Selection},

year={2020},

volume={E103-D},

number={7},

pages={1693-1702},

abstract={This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas *et al.* [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the *L*_{2}-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.},

keywords={},

doi={10.1587/transinf.2019EDP7274},

ISSN={1745-1361},

month={July},}

Copy

TY - JOUR

TI - Stochastic Discrete First-Order Algorithm for Feature Subset Selection

T2 - IEICE TRANSACTIONS on Information

SP - 1693

EP - 1702

AU - Kota KUDO

AU - Yuichi TAKANO

AU - Ryo NOMURA

PY - 2020

DO - 10.1587/transinf.2019EDP7274

JO - IEICE TRANSACTIONS on Information

SN - 1745-1361

VL - E103-D

IS - 7

JA - IEICE TRANSACTIONS on Information

Y1 - July 2020

AB - This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas *et al.* [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the *L*_{2}-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.

ER -