Convolutional Auto-Encoder and Adversarial Domain Adaptation for Cross-Corpus Speech Emotion Recognition

Yang WANG; Hongliang FU; Huawei TAO; Jing YANG; Hongyi GE; Yue XIE

doi:10.1587/transinf.2022EDL8045

Convolutional Auto-Encoder and Adversarial Domain Adaptation for Cross-Corpus Speech Emotion Recognition

Yang WANG, Hongliang FU, Huawei TAO, Jing YANG, Hongyi GE, Yue XIE

Full Text Views

0

Cite this

Summary :

This letter focuses on the cross-corpus speech emotion recognition (SER) task, in which the training and testing speech signals in cross-corpus SER belong to different speech corpora. Existing algorithms are incapable of effectively extracting common sentiment information between different corpora to facilitate knowledge transfer. To address this challenging problem, a novel convolutional auto-encoder and adversarial domain adaptation (CAEADA) framework for cross-corpus SER is proposed. The framework first constructs a one-dimensional convolutional auto-encoder (1D-CAE) for feature processing, which can explore the correlation among adjacent one-dimensional statistic features and the feature representation can be enhanced by the architecture based on encoder-decoder-style. Subsequently the adversarial domain adaptation (ADA) module alleviates the feature distributions discrepancy between the source and target domains by confusing domain discriminator, and specifically employs maximum mean discrepancy (MMD) to better accomplish feature transformation. To evaluate the proposed CAEADA, extensive experiments were conducted on EmoDB, eNTERFACE, and CASIA speech corpora, and the results show that the proposed method outperformed other approaches.

Publication: IEICE TRANSACTIONS on Information Vol.E105-D No.10 pp.1803-1806

Publication Date: 2022/10/01

Publicized: 2022/07/12

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2022EDL8045

Type of Manuscript: LETTER

Category: Artificial Intelligence, Data Mining

Authors

Yang WANG
  Henan University of Technology, Ministry of Education,Henan University of Technology
Hongliang FU
  Henan University of Technology, Ministry of Education,Henan University of Technology
Huawei TAO
  Henan University of Technology, Ministry of Education,Henan University of Technology
Jing YANG
  Henan University of Technology, Ministry of Education,Henan University of Technology
Hongyi GE
  Henan University of Technology, Ministry of Education,Henan University of Technology
Yue XIE
  Nanjing Institute of Technology

Keyword

cross-corpus speech emotion recognition, convolutional auto-encoder, adversarial domain adaptation

Cite this

Copy

Yang WANG, Hongliang FU, Huawei TAO, Jing YANG, Hongyi GE, Yue XIE, "Convolutional Auto-Encoder and Adversarial Domain Adaptation for Cross-Corpus Speech Emotion Recognition" in IEICE TRANSACTIONS on Information, vol. E105-D, no. 10, pp. 1803-1806, October 2022, doi: 10.1587/transinf.2022EDL8045.
Abstract: This letter focuses on the cross-corpus speech emotion recognition (SER) task, in which the training and testing speech signals in cross-corpus SER belong to different speech corpora. Existing algorithms are incapable of effectively extracting common sentiment information between different corpora to facilitate knowledge transfer. To address this challenging problem, a novel convolutional auto-encoder and adversarial domain adaptation (CAEADA) framework for cross-corpus SER is proposed. The framework first constructs a one-dimensional convolutional auto-encoder (1D-CAE) for feature processing, which can explore the correlation among adjacent one-dimensional statistic features and the feature representation can be enhanced by the architecture based on encoder-decoder-style. Subsequently the adversarial domain adaptation (ADA) module alleviates the feature distributions discrepancy between the source and target domains by confusing domain discriminator, and specifically employs maximum mean discrepancy (MMD) to better accomplish feature transformation. To evaluate the proposed CAEADA, extensive experiments were conducted on EmoDB, eNTERFACE, and CASIA speech corpora, and the results show that the proposed method outperformed other approaches.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2022EDL8045/_p

Copy

@ARTICLE{e105-d_10_1803,
author={Yang WANG, Hongliang FU, Huawei TAO, Jing YANG, Hongyi GE, Yue XIE, },
journal={IEICE TRANSACTIONS on Information},
title={Convolutional Auto-Encoder and Adversarial Domain Adaptation for Cross-Corpus Speech Emotion Recognition},
year={2022},
volume={E105-D},
number={10},
pages={1803-1806},
abstract={This letter focuses on the cross-corpus speech emotion recognition (SER) task, in which the training and testing speech signals in cross-corpus SER belong to different speech corpora. Existing algorithms are incapable of effectively extracting common sentiment information between different corpora to facilitate knowledge transfer. To address this challenging problem, a novel convolutional auto-encoder and adversarial domain adaptation (CAEADA) framework for cross-corpus SER is proposed. The framework first constructs a one-dimensional convolutional auto-encoder (1D-CAE) for feature processing, which can explore the correlation among adjacent one-dimensional statistic features and the feature representation can be enhanced by the architecture based on encoder-decoder-style. Subsequently the adversarial domain adaptation (ADA) module alleviates the feature distributions discrepancy between the source and target domains by confusing domain discriminator, and specifically employs maximum mean discrepancy (MMD) to better accomplish feature transformation. To evaluate the proposed CAEADA, extensive experiments were conducted on EmoDB, eNTERFACE, and CASIA speech corpora, and the results show that the proposed method outperformed other approaches.},
keywords={},
doi={10.1587/transinf.2022EDL8045},
ISSN={1745-1361},
month={October},}

Copy

TY - JOUR
TI - Convolutional Auto-Encoder and Adversarial Domain Adaptation for Cross-Corpus Speech Emotion Recognition
T2 - IEICE TRANSACTIONS on Information
SP - 1803
EP - 1806
AU - Yang WANG
AU - Hongliang FU
AU - Huawei TAO
AU - Jing YANG
AU - Hongyi GE
AU - Yue XIE
PY - 2022
DO - 10.1587/transinf.2022EDL8045
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E105-D
IS - 10
JA - IEICE TRANSACTIONS on Information
Y1 - October 2022
AB - This letter focuses on the cross-corpus speech emotion recognition (SER) task, in which the training and testing speech signals in cross-corpus SER belong to different speech corpora. Existing algorithms are incapable of effectively extracting common sentiment information between different corpora to facilitate knowledge transfer. To address this challenging problem, a novel convolutional auto-encoder and adversarial domain adaptation (CAEADA) framework for cross-corpus SER is proposed. The framework first constructs a one-dimensional convolutional auto-encoder (1D-CAE) for feature processing, which can explore the correlation among adjacent one-dimensional statistic features and the feature representation can be enhanced by the architecture based on encoder-decoder-style. Subsequently the adversarial domain adaptation (ADA) module alleviates the feature distributions discrepancy between the source and target domains by confusing domain discriminator, and specifically employs maximum mean discrepancy (MMD) to better accomplish feature transformation. To evaluate the proposed CAEADA, extensive experiments were conducted on EmoDB, eNTERFACE, and CASIA speech corpora, and the results show that the proposed method outperformed other approaches.
ER -