The search functionality is under construction.
The search functionality is under construction.

A Multitask Learning Approach Based on Cascaded Attention Network and Self-Adaption Loss for Speech Emotion Recognition

Yang LIU, Yuqi XIA, Haoqin SUN, Xiaolei MENG, Jianxiong BAI, Wenbo GUAN, Zhen ZHAO, Yongwei LI

  • Full Text Views

    16

  • Cite this

Summary :

Speech emotion recognition (SER) has been a complex and difficult task for a long time due to emotional complexity. In this paper, we propose a multitask deep learning approach based on cascaded attention network and self-adaption loss for SER. First, non-personalized features are extracted to represent the process of emotion change while reducing external variables' influence. Second, to highlight salient speech emotion features, a cascade attention network is proposed, where spatial temporal attention can effectively locate the regions of speech that express emotion, while self-attention reduces the dependence on external information. Finally, the influence brought by the differences in gender and human perception of external information is alleviated by using a multitask learning strategy, where a self-adaption loss is introduced to determine the weights of different tasks dynamically. Experimental results on IEMOCAP dataset demonstrate that our method gains an absolute improvement of 1.97% and 0.91% over state-of-the-art strategies in terms of weighted accuracy (WA) and unweighted accuracy (UA), respectively.

Publication
IEICE TRANSACTIONS on Fundamentals Vol.E106-A No.6 pp.876-885
Publication Date
2023/06/01
Publicized
2022/12/08
Online ISSN
1745-1337
DOI
10.1587/transfun.2022EAP1091
Type of Manuscript
PAPER
Category
Speech and Hearing

Authors

Yang LIU
  Qingdao University of Science and Technology
Yuqi XIA
  Qingdao University of Science and Technology
Haoqin SUN
  Qingdao University of Science and Technology
Xiaolei MENG
  Qingdao University of Science and Technology
Jianxiong BAI
  Qingdao University of Science and Technology
Wenbo GUAN
  Qingdao University of Science and Technology
Zhen ZHAO
  Qingdao University of Science and Technology
Yongwei LI
  Chinese Academy of Sciences

Keyword