Feature Selection by Computing Mutual Information Based on Partitions

Chengxiang YIN; Hongjun ZHANG; Rui ZHANG; Zilin ZENG; Xiuli QI; Yuntian FENG

doi:10.1587/transinf.2017EDP7250

Feature Selection by Computing Mutual Information Based on Partitions

Chengxiang YIN, Hongjun ZHANG, Rui ZHANG, Zilin ZENG, Xiuli QI, Yuntian FENG

Full Text Views

0

Cite this

Summary :

The main idea of filter methods in feature selection is constructing a feature-assessing criterion and searching for feature subset that optimizes the criterion. The primary principle of designing such criterion is to capture the relevance between feature subset and the class as precisely as possible. It would be difficult to compute the relevance directly due to the computation complexity when the size of feature subset grows. As a result, researchers adopt approximate strategies to measure relevance. Though these strategies worked well in some applications, they suffer from three problems: parameter determination problem, the neglect of feature interaction information and overestimation of some features. We propose a new feature selection algorithm that could compute mutual information between feature subset and the class directly without deteriorating computation complexity based on the computation of partitions. In light of the specific properties of mutual information and partitions, we propose a pruning rule and a stopping criterion to accelerate the searching speed. To evaluate the effectiveness of the proposed algorithm, we compare our algorithm to the other five algorithms in terms of the number of selected features and the classification accuracies on three classifiers. The results on the six synthetic datasets show that our algorithm performs well in capturing interaction information. The results on the thirteen real world datasets show that our algorithm selects less yet better feature subset.

Publication: IEICE TRANSACTIONS on Information Vol.E101-D No.2 pp.437-446

Publication Date: 2018/02/01

Publicized: 2017/11/01

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2017EDP7250

Type of Manuscript: PAPER

Category: Artificial Intelligence, Data Mining

Authors

Chengxiang YIN
  Army Engineering University of PLA
Hongjun ZHANG
  Army Engineering University of PLA
Rui ZHANG
  Army Engineering University of PLA
Zilin ZENG
  Army Infantry College of PLA
Xiuli QI
  Army Engineering University of PLA
Yuntian FENG
  Army Engineering University of PLA

Keyword

partition, feature selection, feature interaction, FSMIP

Cite this

Copy

Chengxiang YIN, Hongjun ZHANG, Rui ZHANG, Zilin ZENG, Xiuli QI, Yuntian FENG, "Feature Selection by Computing Mutual Information Based on Partitions" in IEICE TRANSACTIONS on Information, vol. E101-D, no. 2, pp. 437-446, February 2018, doi: 10.1587/transinf.2017EDP7250.
Abstract: The main idea of filter methods in feature selection is constructing a feature-assessing criterion and searching for feature subset that optimizes the criterion. The primary principle of designing such criterion is to capture the relevance between feature subset and the class as precisely as possible. It would be difficult to compute the relevance directly due to the computation complexity when the size of feature subset grows. As a result, researchers adopt approximate strategies to measure relevance. Though these strategies worked well in some applications, they suffer from three problems: parameter determination problem, the neglect of feature interaction information and overestimation of some features. We propose a new feature selection algorithm that could compute mutual information between feature subset and the class directly without deteriorating computation complexity based on the computation of partitions. In light of the specific properties of mutual information and partitions, we propose a pruning rule and a stopping criterion to accelerate the searching speed. To evaluate the effectiveness of the proposed algorithm, we compare our algorithm to the other five algorithms in terms of the number of selected features and the classification accuracies on three classifiers. The results on the six synthetic datasets show that our algorithm performs well in capturing interaction information. The results on the thirteen real world datasets show that our algorithm selects less yet better feature subset.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2017EDP7250/_p

Copy

@ARTICLE{e101-d_2_437,
author={Chengxiang YIN, Hongjun ZHANG, Rui ZHANG, Zilin ZENG, Xiuli QI, Yuntian FENG, },
journal={IEICE TRANSACTIONS on Information},
title={Feature Selection by Computing Mutual Information Based on Partitions},
year={2018},
volume={E101-D},
number={2},
pages={437-446},
abstract={The main idea of filter methods in feature selection is constructing a feature-assessing criterion and searching for feature subset that optimizes the criterion. The primary principle of designing such criterion is to capture the relevance between feature subset and the class as precisely as possible. It would be difficult to compute the relevance directly due to the computation complexity when the size of feature subset grows. As a result, researchers adopt approximate strategies to measure relevance. Though these strategies worked well in some applications, they suffer from three problems: parameter determination problem, the neglect of feature interaction information and overestimation of some features. We propose a new feature selection algorithm that could compute mutual information between feature subset and the class directly without deteriorating computation complexity based on the computation of partitions. In light of the specific properties of mutual information and partitions, we propose a pruning rule and a stopping criterion to accelerate the searching speed. To evaluate the effectiveness of the proposed algorithm, we compare our algorithm to the other five algorithms in terms of the number of selected features and the classification accuracies on three classifiers. The results on the six synthetic datasets show that our algorithm performs well in capturing interaction information. The results on the thirteen real world datasets show that our algorithm selects less yet better feature subset.},
keywords={},
doi={10.1587/transinf.2017EDP7250},
ISSN={1745-1361},
month={February},}

Copy

TY - JOUR
TI - Feature Selection by Computing Mutual Information Based on Partitions
T2 - IEICE TRANSACTIONS on Information
SP - 437
EP - 446
AU - Chengxiang YIN
AU - Hongjun ZHANG
AU - Rui ZHANG
AU - Zilin ZENG
AU - Xiuli QI
AU - Yuntian FENG
PY - 2018
DO - 10.1587/transinf.2017EDP7250
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E101-D
IS - 2
JA - IEICE TRANSACTIONS on Information
Y1 - February 2018
AB - The main idea of filter methods in feature selection is constructing a feature-assessing criterion and searching for feature subset that optimizes the criterion. The primary principle of designing such criterion is to capture the relevance between feature subset and the class as precisely as possible. It would be difficult to compute the relevance directly due to the computation complexity when the size of feature subset grows. As a result, researchers adopt approximate strategies to measure relevance. Though these strategies worked well in some applications, they suffer from three problems: parameter determination problem, the neglect of feature interaction information and overestimation of some features. We propose a new feature selection algorithm that could compute mutual information between feature subset and the class directly without deteriorating computation complexity based on the computation of partitions. In light of the specific properties of mutual information and partitions, we propose a pruning rule and a stopping criterion to accelerate the searching speed. To evaluate the effectiveness of the proposed algorithm, we compare our algorithm to the other five algorithms in terms of the number of selected features and the classification accuracies on three classifiers. The results on the six synthetic datasets show that our algorithm performs well in capturing interaction information. The results on the thirteen real world datasets show that our algorithm selects less yet better feature subset.
ER -