IEICE TRANSACTIONS on Information
Model Shrinkage for Discriminative Language Models
Summary :
This paper describes a technique for overcoming the model shrinkage problem in automatic speech recognition (ASR), which allows application developers and users to control the model size with less degradation of accuracy. Recently, models for ASR systems tend to be large and this can constitute a bottleneck for developers and users without special knowledge of ASR with respect to introducing the ASR function. Specifically, discriminative language models (DLMs) are usually designed in a high-dimensional parameter space, although DLMs have gained increasing attention as an approach for improving recognition accuracy. Our proposed method can be applied to linear models including DLMs, in which the score of an input sample is given by the inner product of its features and the model parameters, but our proposed method can shrink models in an easy computation by obtaining simple statistics, which are square sums of feature values appearing in a data set. Our experimental results show that our proposed method can shrink a DLM with little degradation in accuracy and perform properly whether or not the data for obtaining the statistics are the same as the data for training the model.
- Publication
- IEICE TRANSACTIONS on Information Vol.E95-D No.5 pp.1465-1474
- Publication Date
- 2012/05/01
- Publicized
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.E95.D.1465
- Type of Manuscript
- PAPER
- Category
- Speech and Hearing
Authors
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Takanobu OBA, Takaaki HORI, Atsushi NAKAMURA, Akinori ITO, "Model Shrinkage for Discriminative Language Models" in IEICE TRANSACTIONS on Information,
vol. E95-D, no. 5, pp. 1465-1474, May 2012, doi: 10.1587/transinf.E95.D.1465.
Abstract: This paper describes a technique for overcoming the model shrinkage problem in automatic speech recognition (ASR), which allows application developers and users to control the model size with less degradation of accuracy. Recently, models for ASR systems tend to be large and this can constitute a bottleneck for developers and users without special knowledge of ASR with respect to introducing the ASR function. Specifically, discriminative language models (DLMs) are usually designed in a high-dimensional parameter space, although DLMs have gained increasing attention as an approach for improving recognition accuracy. Our proposed method can be applied to linear models including DLMs, in which the score of an input sample is given by the inner product of its features and the model parameters, but our proposed method can shrink models in an easy computation by obtaining simple statistics, which are square sums of feature values appearing in a data set. Our experimental results show that our proposed method can shrink a DLM with little degradation in accuracy and perform properly whether or not the data for obtaining the statistics are the same as the data for training the model.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.E95.D.1465/_p
Copy
@ARTICLE{e95-d_5_1465,
author={Takanobu OBA, Takaaki HORI, Atsushi NAKAMURA, Akinori ITO, },
journal={IEICE TRANSACTIONS on Information},
title={Model Shrinkage for Discriminative Language Models},
year={2012},
volume={E95-D},
number={5},
pages={1465-1474},
abstract={This paper describes a technique for overcoming the model shrinkage problem in automatic speech recognition (ASR), which allows application developers and users to control the model size with less degradation of accuracy. Recently, models for ASR systems tend to be large and this can constitute a bottleneck for developers and users without special knowledge of ASR with respect to introducing the ASR function. Specifically, discriminative language models (DLMs) are usually designed in a high-dimensional parameter space, although DLMs have gained increasing attention as an approach for improving recognition accuracy. Our proposed method can be applied to linear models including DLMs, in which the score of an input sample is given by the inner product of its features and the model parameters, but our proposed method can shrink models in an easy computation by obtaining simple statistics, which are square sums of feature values appearing in a data set. Our experimental results show that our proposed method can shrink a DLM with little degradation in accuracy and perform properly whether or not the data for obtaining the statistics are the same as the data for training the model.},
keywords={},
doi={10.1587/transinf.E95.D.1465},
ISSN={1745-1361},
month={May},}
Copy
TY - JOUR
TI - Model Shrinkage for Discriminative Language Models
T2 - IEICE TRANSACTIONS on Information
SP - 1465
EP - 1474
AU - Takanobu OBA
AU - Takaaki HORI
AU - Atsushi NAKAMURA
AU - Akinori ITO
PY - 2012
DO - 10.1587/transinf.E95.D.1465
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E95-D
IS - 5
JA - IEICE TRANSACTIONS on Information
Y1 - May 2012
AB - This paper describes a technique for overcoming the model shrinkage problem in automatic speech recognition (ASR), which allows application developers and users to control the model size with less degradation of accuracy. Recently, models for ASR systems tend to be large and this can constitute a bottleneck for developers and users without special knowledge of ASR with respect to introducing the ASR function. Specifically, discriminative language models (DLMs) are usually designed in a high-dimensional parameter space, although DLMs have gained increasing attention as an approach for improving recognition accuracy. Our proposed method can be applied to linear models including DLMs, in which the score of an input sample is given by the inner product of its features and the model parameters, but our proposed method can shrink models in an easy computation by obtaining simple statistics, which are square sums of feature values appearing in a data set. Our experimental results show that our proposed method can shrink a DLM with little degradation in accuracy and perform properly whether or not the data for obtaining the statistics are the same as the data for training the model.
ER -
English
