IEICE TRANSACTIONS on Information
An Objective Method for Evaluating Speech Translation System: Using a Second Language Learner's Corpus
Summary :
In this paper we propose an objective method for assessing the capability of a speech translation system. It automates the translation paired comparison method, which gives a simple, easy to understand TOEIC score proposed by Sugaya et al., to succinctly evaluate a speech translation system. To avoid the expensive evaluation cost of the original method where large manual effort is required, the new objective method automates the procedure by employing an objective metric such as BLEU and DP-based measure. The evaluation results obtained by the proposed method are similar to those of the original method. Also, the proposed method is used to evaluate the usefulness of a speech translation system. It is then found that our speech translation system is useful in general, even to users with higher TOEIC score than the system's.
- Publication
- IEICE TRANSACTIONS on Information Vol.E88-D No.3 pp.569-577
- Publication Date
- 2005/03/01
- Publicized
- Online ISSN
- DOI
- 10.1093/ietisy/e88-d.3.569
- Type of Manuscript
- Special Section PAPER (Special Section on Corpus-Based Speech Technologies)
- Category
- Speech Corpora and Related Topics
Authors
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Keiji YASUDA, Fumiaki SUGAYA, Toshiyuki TAKEZAWA, Genichiro KIKUI, Seiichi YAMAMOTO, Masuzo YANAGIDA, "An Objective Method for Evaluating Speech Translation System: Using a Second Language Learner's Corpus" in IEICE TRANSACTIONS on Information,
vol. E88-D, no. 3, pp. 569-577, March 2005, doi: 10.1093/ietisy/e88-d.3.569.
Abstract: In this paper we propose an objective method for assessing the capability of a speech translation system. It automates the translation paired comparison method, which gives a simple, easy to understand TOEIC score proposed by Sugaya et al., to succinctly evaluate a speech translation system. To avoid the expensive evaluation cost of the original method where large manual effort is required, the new objective method automates the procedure by employing an objective metric such as BLEU and DP-based measure. The evaluation results obtained by the proposed method are similar to those of the original method. Also, the proposed method is used to evaluate the usefulness of a speech translation system. It is then found that our speech translation system is useful in general, even to users with higher TOEIC score than the system's.
URL: https://global.ieice.org/en_transactions/information/10.1093/ietisy/e88-d.3.569/_p
Copy
@ARTICLE{e88-d_3_569,
author={Keiji YASUDA, Fumiaki SUGAYA, Toshiyuki TAKEZAWA, Genichiro KIKUI, Seiichi YAMAMOTO, Masuzo YANAGIDA, },
journal={IEICE TRANSACTIONS on Information},
title={An Objective Method for Evaluating Speech Translation System: Using a Second Language Learner's Corpus},
year={2005},
volume={E88-D},
number={3},
pages={569-577},
abstract={In this paper we propose an objective method for assessing the capability of a speech translation system. It automates the translation paired comparison method, which gives a simple, easy to understand TOEIC score proposed by Sugaya et al., to succinctly evaluate a speech translation system. To avoid the expensive evaluation cost of the original method where large manual effort is required, the new objective method automates the procedure by employing an objective metric such as BLEU and DP-based measure. The evaluation results obtained by the proposed method are similar to those of the original method. Also, the proposed method is used to evaluate the usefulness of a speech translation system. It is then found that our speech translation system is useful in general, even to users with higher TOEIC score than the system's.},
keywords={},
doi={10.1093/ietisy/e88-d.3.569},
ISSN={},
month={March},}
Copy
TY - JOUR
TI - An Objective Method for Evaluating Speech Translation System: Using a Second Language Learner's Corpus
T2 - IEICE TRANSACTIONS on Information
SP - 569
EP - 577
AU - Keiji YASUDA
AU - Fumiaki SUGAYA
AU - Toshiyuki TAKEZAWA
AU - Genichiro KIKUI
AU - Seiichi YAMAMOTO
AU - Masuzo YANAGIDA
PY - 2005
DO - 10.1093/ietisy/e88-d.3.569
JO - IEICE TRANSACTIONS on Information
SN -
VL - E88-D
IS - 3
JA - IEICE TRANSACTIONS on Information
Y1 - March 2005
AB - In this paper we propose an objective method for assessing the capability of a speech translation system. It automates the translation paired comparison method, which gives a simple, easy to understand TOEIC score proposed by Sugaya et al., to succinctly evaluate a speech translation system. To avoid the expensive evaluation cost of the original method where large manual effort is required, the new objective method automates the procedure by employing an objective metric such as BLEU and DP-based measure. The evaluation results obtained by the proposed method are similar to those of the original method. Also, the proposed method is used to evaluate the usefulness of a speech translation system. It is then found that our speech translation system is useful in general, even to users with higher TOEIC score than the system's.
ER -
English
