Structured Adaptive Regularization of Weight Vectors for a Robust Grapheme-to-Phoneme Conversion Model
Summary :
Grapheme-to-phoneme (g2p) conversion, used to estimate the pronunciations of out-of-vocabulary (OOV) words, is a highly important part of recognition systems, as well as text-to-speech systems. The current state-of-the-art approach in g2p conversion is structured learning based on the Margin Infused Relaxed Algorithm (MIRA), which is an online discriminative training method for multiclass classification. However, it is known that the aggressive weight update method of MIRA is prone to overfitting, even if the current example is an outlier or noisy. Adaptive Regularization of Weight Vectors (AROW) has been proposed to resolve this problem for binary classification. In addition, AROW's update rule is simpler and more efficient than that of MIRA, allowing for more efficient training. Although AROW has these advantages, it has not been applied to g2p conversion yet. In this paper, we first apply AROW on g2p conversion task which is structured learning problem. In an evaluation that employed a dataset generated from the collective knowledge on the Web, our proposed approach achieves a 6.8% error reduction rate compared to MIRA in terms of phoneme error rate. Also the learning time of our proposed approach was shorter than that of MIRA in almost datasets.
- Publication
- IEICE TRANSACTIONS on Information Vol.E97-D No.6 pp.1468-1476
- Publication Date
- 2014/06/01
- Publicized
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.E97.D.1468
- Type of Manuscript
- Special Section PAPER (Special Section on Advances in Modeling for Real-world Speech Information Processing and its Application)
- Category
- Speech Synthesis and Related Topics
Authors
Keigo KUBO
Nara Institute of Science and Technology
Sakriani SAKTI
Nara Institute of Science and Technology
Graham NEUBIG
Nara Institute of Science and Technology
Tomoki TODA
Nara Institute of Science and Technology
Satoshi NAKAMURA
Nara Institute of Science and Technology
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Keigo KUBO, Sakriani SAKTI, Graham NEUBIG, Tomoki TODA, Satoshi NAKAMURA, "Structured Adaptive Regularization of Weight Vectors for a Robust Grapheme-to-Phoneme Conversion Model" in IEICE TRANSACTIONS on Information,
vol. E97-D, no. 6, pp. 1468-1476, June 2014, doi: 10.1587/transinf.E97.D.1468.
Abstract: Grapheme-to-phoneme (g2p) conversion, used to estimate the pronunciations of out-of-vocabulary (OOV) words, is a highly important part of recognition systems, as well as text-to-speech systems. The current state-of-the-art approach in g2p conversion is structured learning based on the Margin Infused Relaxed Algorithm (MIRA), which is an online discriminative training method for multiclass classification. However, it is known that the aggressive weight update method of MIRA is prone to overfitting, even if the current example is an outlier or noisy. Adaptive Regularization of Weight Vectors (AROW) has been proposed to resolve this problem for binary classification. In addition, AROW's update rule is simpler and more efficient than that of MIRA, allowing for more efficient training. Although AROW has these advantages, it has not been applied to g2p conversion yet. In this paper, we first apply AROW on g2p conversion task which is structured learning problem. In an evaluation that employed a dataset generated from the collective knowledge on the Web, our proposed approach achieves a 6.8% error reduction rate compared to MIRA in terms of phoneme error rate. Also the learning time of our proposed approach was shorter than that of MIRA in almost datasets.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.E97.D.1468/_p
Copy
@ARTICLE{e97-d_6_1468,
author={Keigo KUBO, Sakriani SAKTI, Graham NEUBIG, Tomoki TODA, Satoshi NAKAMURA, },
journal={IEICE TRANSACTIONS on Information},
title={Structured Adaptive Regularization of Weight Vectors for a Robust Grapheme-to-Phoneme Conversion Model},
year={2014},
volume={E97-D},
number={6},
pages={1468-1476},
abstract={Grapheme-to-phoneme (g2p) conversion, used to estimate the pronunciations of out-of-vocabulary (OOV) words, is a highly important part of recognition systems, as well as text-to-speech systems. The current state-of-the-art approach in g2p conversion is structured learning based on the Margin Infused Relaxed Algorithm (MIRA), which is an online discriminative training method for multiclass classification. However, it is known that the aggressive weight update method of MIRA is prone to overfitting, even if the current example is an outlier or noisy. Adaptive Regularization of Weight Vectors (AROW) has been proposed to resolve this problem for binary classification. In addition, AROW's update rule is simpler and more efficient than that of MIRA, allowing for more efficient training. Although AROW has these advantages, it has not been applied to g2p conversion yet. In this paper, we first apply AROW on g2p conversion task which is structured learning problem. In an evaluation that employed a dataset generated from the collective knowledge on the Web, our proposed approach achieves a 6.8% error reduction rate compared to MIRA in terms of phoneme error rate. Also the learning time of our proposed approach was shorter than that of MIRA in almost datasets.},
keywords={},
doi={10.1587/transinf.E97.D.1468},
ISSN={1745-1361},
month={June},}
Copy
TY - JOUR
TI - Structured Adaptive Regularization of Weight Vectors for a Robust Grapheme-to-Phoneme Conversion Model
T2 - IEICE TRANSACTIONS on Information
SP - 1468
EP - 1476
AU - Keigo KUBO
AU - Sakriani SAKTI
AU - Graham NEUBIG
AU - Tomoki TODA
AU - Satoshi NAKAMURA
PY - 2014
DO - 10.1587/transinf.E97.D.1468
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E97-D
IS - 6
JA - IEICE TRANSACTIONS on Information
Y1 - June 2014
AB - Grapheme-to-phoneme (g2p) conversion, used to estimate the pronunciations of out-of-vocabulary (OOV) words, is a highly important part of recognition systems, as well as text-to-speech systems. The current state-of-the-art approach in g2p conversion is structured learning based on the Margin Infused Relaxed Algorithm (MIRA), which is an online discriminative training method for multiclass classification. However, it is known that the aggressive weight update method of MIRA is prone to overfitting, even if the current example is an outlier or noisy. Adaptive Regularization of Weight Vectors (AROW) has been proposed to resolve this problem for binary classification. In addition, AROW's update rule is simpler and more efficient than that of MIRA, allowing for more efficient training. Although AROW has these advantages, it has not been applied to g2p conversion yet. In this paper, we first apply AROW on g2p conversion task which is structured learning problem. In an evaluation that employed a dataset generated from the collective knowledge on the Web, our proposed approach achieves a 6.8% error reduction rate compared to MIRA in terms of phoneme error rate. Also the learning time of our proposed approach was shorter than that of MIRA in almost datasets.
ER -
English
