A Joint Neural Model for Fine-Grained Named Entity Classification of Wikipedia Articles

Masatoshi SUZUKI; Koji MATSUDA; Satoshi SEKINE; Naoaki OKAZAKI; Kentaro INUI

doi:10.1587/transinf.2017SWP0005

A Joint Neural Model for Fine-Grained Named Entity Classification of Wikipedia Articles

Masatoshi SUZUKI, Koji MATSUDA, Satoshi SEKINE, Naoaki OKAZAKI, Kentaro INUI

Full Text Views

0

Cite this

Summary :

This paper addresses the task of assigning labels of fine-grained named entity (NE) types to Wikipedia articles. Information of NE types are useful when extracting knowledge of NEs from natural language text. It is common to apply an approach based on supervised machine learning to named entity classification. However, in a setting of classifying into fine-grained types, one big challenge is how to alleviate the data sparseness problem since one may obtain far fewer instances for each fine-grained types. To address this problem, we propose two methods. First, we introduce a multi-task learning framework, in which NE type classifiers are all jointly trained with a neural network. The neural network has a hidden layer, where we expect that effective combinations of input features are learned across different NE types. Second, we propose to extend the input feature set by exploiting the hyperlink structure of Wikipedia. While most of previous studies are focusing on engineering features from the articles' contents, we observe that the information of the contexts the article is mentioned can also be a useful clue for NE type classification. Concretely, we propose to learn article vectors (i.e. entity embeddings) from Wikipedia's hyperlink structure using a Skip-gram model. Then we incorporate the learned article vectors into the input feature set for NE type classification. To conduct large-scale practical experiments, we created a new dataset containing over 22,000 manually labeled articles. With the dataset, we empirically show that both of our ideas gained their own statistically significant improvement separately in classification accuracy. Moreover, we show that our proposed methods are particularly effective in labeling infrequent NE types. We've made the learned article vectors publicly available. The labeled dataset is available if one contacts the authors.

Publication: IEICE TRANSACTIONS on Information Vol.E101-D No.1 pp.73-81

Publication Date: 2018/01/01

Publicized: 2017/09/15

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2017SWP0005

Type of Manuscript: Special Section PAPER (Special Section on Semantic Web and Linked Data)

Category

Authors

Masatoshi SUZUKI
  Tohoku University
Koji MATSUDA
  Tohoku University
Satoshi SEKINE
  Language Craft Inc,RIKEN Center for Advanced Intelligence Project
Naoaki OKAZAKI
  Tokyo Institute of Technology
Kentaro INUI
  Tohoku University,RIKEN Center for Advanced Intelligence Project

Keyword

named entity classification, wikipedia, multi-task learning, neural network

Cite this

Copy

Masatoshi SUZUKI, Koji MATSUDA, Satoshi SEKINE, Naoaki OKAZAKI, Kentaro INUI, "A Joint Neural Model for Fine-Grained Named Entity Classification of Wikipedia Articles" in IEICE TRANSACTIONS on Information, vol. E101-D, no. 1, pp. 73-81, January 2018, doi: 10.1587/transinf.2017SWP0005.
Abstract: This paper addresses the task of assigning labels of fine-grained named entity (NE) types to Wikipedia articles. Information of NE types are useful when extracting knowledge of NEs from natural language text. It is common to apply an approach based on supervised machine learning to named entity classification. However, in a setting of classifying into fine-grained types, one big challenge is how to alleviate the data sparseness problem since one may obtain far fewer instances for each fine-grained types. To address this problem, we propose two methods. First, we introduce a multi-task learning framework, in which NE type classifiers are all jointly trained with a neural network. The neural network has a hidden layer, where we expect that effective combinations of input features are learned across different NE types. Second, we propose to extend the input feature set by exploiting the hyperlink structure of Wikipedia. While most of previous studies are focusing on engineering features from the articles' contents, we observe that the information of the contexts the article is mentioned can also be a useful clue for NE type classification. Concretely, we propose to learn article vectors (i.e. entity embeddings) from Wikipedia's hyperlink structure using a Skip-gram model. Then we incorporate the learned article vectors into the input feature set for NE type classification. To conduct large-scale practical experiments, we created a new dataset containing over 22,000 manually labeled articles. With the dataset, we empirically show that both of our ideas gained their own statistically significant improvement separately in classification accuracy. Moreover, we show that our proposed methods are particularly effective in labeling infrequent NE types. We've made the learned article vectors publicly available. The labeled dataset is available if one contacts the authors.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2017SWP0005/_p

Copy

@ARTICLE{e101-d_1_73,
author={Masatoshi SUZUKI, Koji MATSUDA, Satoshi SEKINE, Naoaki OKAZAKI, Kentaro INUI, },
journal={IEICE TRANSACTIONS on Information},
title={A Joint Neural Model for Fine-Grained Named Entity Classification of Wikipedia Articles},
year={2018},
volume={E101-D},
number={1},
pages={73-81},
abstract={This paper addresses the task of assigning labels of fine-grained named entity (NE) types to Wikipedia articles. Information of NE types are useful when extracting knowledge of NEs from natural language text. It is common to apply an approach based on supervised machine learning to named entity classification. However, in a setting of classifying into fine-grained types, one big challenge is how to alleviate the data sparseness problem since one may obtain far fewer instances for each fine-grained types. To address this problem, we propose two methods. First, we introduce a multi-task learning framework, in which NE type classifiers are all jointly trained with a neural network. The neural network has a hidden layer, where we expect that effective combinations of input features are learned across different NE types. Second, we propose to extend the input feature set by exploiting the hyperlink structure of Wikipedia. While most of previous studies are focusing on engineering features from the articles' contents, we observe that the information of the contexts the article is mentioned can also be a useful clue for NE type classification. Concretely, we propose to learn article vectors (i.e. entity embeddings) from Wikipedia's hyperlink structure using a Skip-gram model. Then we incorporate the learned article vectors into the input feature set for NE type classification. To conduct large-scale practical experiments, we created a new dataset containing over 22,000 manually labeled articles. With the dataset, we empirically show that both of our ideas gained their own statistically significant improvement separately in classification accuracy. Moreover, we show that our proposed methods are particularly effective in labeling infrequent NE types. We've made the learned article vectors publicly available. The labeled dataset is available if one contacts the authors.},
keywords={},
doi={10.1587/transinf.2017SWP0005},
ISSN={1745-1361},
month={January},}

Copy

TY - JOUR
TI - A Joint Neural Model for Fine-Grained Named Entity Classification of Wikipedia Articles
T2 - IEICE TRANSACTIONS on Information
SP - 73
EP - 81
AU - Masatoshi SUZUKI
AU - Koji MATSUDA
AU - Satoshi SEKINE
AU - Naoaki OKAZAKI
AU - Kentaro INUI
PY - 2018
DO - 10.1587/transinf.2017SWP0005
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E101-D
IS - 1
JA - IEICE TRANSACTIONS on Information
Y1 - January 2018
AB - This paper addresses the task of assigning labels of fine-grained named entity (NE) types to Wikipedia articles. Information of NE types are useful when extracting knowledge of NEs from natural language text. It is common to apply an approach based on supervised machine learning to named entity classification. However, in a setting of classifying into fine-grained types, one big challenge is how to alleviate the data sparseness problem since one may obtain far fewer instances for each fine-grained types. To address this problem, we propose two methods. First, we introduce a multi-task learning framework, in which NE type classifiers are all jointly trained with a neural network. The neural network has a hidden layer, where we expect that effective combinations of input features are learned across different NE types. Second, we propose to extend the input feature set by exploiting the hyperlink structure of Wikipedia. While most of previous studies are focusing on engineering features from the articles' contents, we observe that the information of the contexts the article is mentioned can also be a useful clue for NE type classification. Concretely, we propose to learn article vectors (i.e. entity embeddings) from Wikipedia's hyperlink structure using a Skip-gram model. Then we incorporate the learned article vectors into the input feature set for NE type classification. To conduct large-scale practical experiments, we created a new dataset containing over 22,000 manually labeled articles. With the dataset, we empirically show that both of our ideas gained their own statistically significant improvement separately in classification accuracy. Moreover, we show that our proposed methods are particularly effective in labeling infrequent NE types. We've made the learned article vectors publicly available. The labeled dataset is available if one contacts the authors.
ER -