Investigation of Using Continuous Representation of Various Linguistic Units in Neural Network Based Text-to-Speech Synthesis

Xin WANG; Shinji TAKAKI; Junichi YAMAGISHI

doi:10.1587/transinf.2016SLP0011

Investigation of Using Continuous Representation of Various Linguistic Units in Neural Network Based Text-to-Speech Synthesis

Xin WANG, Shinji TAKAKI, Junichi YAMAGISHI

Full Text Views

0

Cite this

Summary :

Building high-quality text-to-speech (TTS) systems without expert knowledge of the target language and/or time-consuming manual annotation of speech and text data is an important yet challenging research topic. In this kind of TTS system, it is vital to find representation of the input text that is both effective and easy to acquire. Recently, the continuous representation of raw word inputs, called “word embedding”, has been successfully used in various natural language processing tasks. It has also been used as the additional or alternative linguistic input features to a neural-network-based acoustic model for TTS systems. In this paper, we further investigate the use of this embedding technique to represent phonemes, syllables and phrases for the acoustic model based on the recurrent and feed-forward neural network. Results of the experiments show that most of these continuous representations cannot significantly improve the system's performance when they are fed into the acoustic model either as additional component or as a replacement of the conventional prosodic context. However, subjective evaluation shows that the continuous representation of phrases can achieve significant improvement when it is combined with the prosodic context as input to the acoustic model based on the feed-forward neural network.

Publication: IEICE TRANSACTIONS on Information Vol.E99-D No.10 pp.2471-2480

Publication Date: 2016/10/01

Publicized: 2016/07/19

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2016SLP0011

Type of Manuscript: Special Section PAPER (Special Section on Recent Advances in Machine Learning for Spoken Language Processing)

Category: Speech synthesis

Authors

Xin WANG
  National Institute of Informatics,SOKENDAI
Shinji TAKAKI
  National Institute of Informatics
Junichi YAMAGISHI
  National Institute of Informatics,SOKENDAI,University of Edinburgh

Keyword

text-to-speech, speech synthesis, recurrent neural network, contexts, word embedding

Cite this

Copy

Xin WANG, Shinji TAKAKI, Junichi YAMAGISHI, "Investigation of Using Continuous Representation of Various Linguistic Units in Neural Network Based Text-to-Speech Synthesis" in IEICE TRANSACTIONS on Information, vol. E99-D, no. 10, pp. 2471-2480, October 2016, doi: 10.1587/transinf.2016SLP0011.
Abstract: Building high-quality text-to-speech (TTS) systems without expert knowledge of the target language and/or time-consuming manual annotation of speech and text data is an important yet challenging research topic. In this kind of TTS system, it is vital to find representation of the input text that is both effective and easy to acquire. Recently, the continuous representation of raw word inputs, called “word embedding”, has been successfully used in various natural language processing tasks. It has also been used as the additional or alternative linguistic input features to a neural-network-based acoustic model for TTS systems. In this paper, we further investigate the use of this embedding technique to represent phonemes, syllables and phrases for the acoustic model based on the recurrent and feed-forward neural network. Results of the experiments show that most of these continuous representations cannot significantly improve the system's performance when they are fed into the acoustic model either as additional component or as a replacement of the conventional prosodic context. However, subjective evaluation shows that the continuous representation of phrases can achieve significant improvement when it is combined with the prosodic context as input to the acoustic model based on the feed-forward neural network.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2016SLP0011/_p

Copy

@ARTICLE{e99-d_10_2471,
author={Xin WANG, Shinji TAKAKI, Junichi YAMAGISHI, },
journal={IEICE TRANSACTIONS on Information},
title={Investigation of Using Continuous Representation of Various Linguistic Units in Neural Network Based Text-to-Speech Synthesis},
year={2016},
volume={E99-D},
number={10},
pages={2471-2480},
abstract={Building high-quality text-to-speech (TTS) systems without expert knowledge of the target language and/or time-consuming manual annotation of speech and text data is an important yet challenging research topic. In this kind of TTS system, it is vital to find representation of the input text that is both effective and easy to acquire. Recently, the continuous representation of raw word inputs, called “word embedding”, has been successfully used in various natural language processing tasks. It has also been used as the additional or alternative linguistic input features to a neural-network-based acoustic model for TTS systems. In this paper, we further investigate the use of this embedding technique to represent phonemes, syllables and phrases for the acoustic model based on the recurrent and feed-forward neural network. Results of the experiments show that most of these continuous representations cannot significantly improve the system's performance when they are fed into the acoustic model either as additional component or as a replacement of the conventional prosodic context. However, subjective evaluation shows that the continuous representation of phrases can achieve significant improvement when it is combined with the prosodic context as input to the acoustic model based on the feed-forward neural network.},
keywords={},
doi={10.1587/transinf.2016SLP0011},
ISSN={1745-1361},
month={October},}

Copy

TY - JOUR
TI - Investigation of Using Continuous Representation of Various Linguistic Units in Neural Network Based Text-to-Speech Synthesis
T2 - IEICE TRANSACTIONS on Information
SP - 2471
EP - 2480
AU - Xin WANG
AU - Shinji TAKAKI
AU - Junichi YAMAGISHI
PY - 2016
DO - 10.1587/transinf.2016SLP0011
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E99-D
IS - 10
JA - IEICE TRANSACTIONS on Information
Y1 - October 2016
AB - Building high-quality text-to-speech (TTS) systems without expert knowledge of the target language and/or time-consuming manual annotation of speech and text data is an important yet challenging research topic. In this kind of TTS system, it is vital to find representation of the input text that is both effective and easy to acquire. Recently, the continuous representation of raw word inputs, called “word embedding”, has been successfully used in various natural language processing tasks. It has also been used as the additional or alternative linguistic input features to a neural-network-based acoustic model for TTS systems. In this paper, we further investigate the use of this embedding technique to represent phonemes, syllables and phrases for the acoustic model based on the recurrent and feed-forward neural network. Results of the experiments show that most of these continuous representations cannot significantly improve the system's performance when they are fed into the acoustic model either as additional component or as a replacement of the conventional prosodic context. However, subjective evaluation shows that the continuous representation of phrases can achieve significant improvement when it is combined with the prosodic context as input to the acoustic model based on the feed-forward neural network.
ER -