Multimodal Prediction of Social Responsiveness Score with BERT-Based Text Features

Takeshi SAGA; Hiroki TANAKA; Hidemi IWASAKA; Satoshi NAKAMURA

doi:10.1587/transinf.2021HCP0009

Multimodal Prediction of Social Responsiveness Score with BERT-Based Text Features

Takeshi SAGA, Hiroki TANAKA, Hidemi IWASAKA, Satoshi NAKAMURA

Full Text Views

0

Cite this

Summary :

Social Skills Training (SST) has been used for years to improve individuals' social skills toward building a better daily life. In SST carried out by humans, the social skills level is usually evaluated through a verbal interview conducted by the trainer. Although this evaluation is based on psychiatric knowledge and professional experience, its quality depends on the trainer's capabilities. Therefore, to standardize such evaluations, quantifiable metrics are required. To meet this need, the second edition of the Social Responsiveness Scale (SRS-2) offers a viable solution because it has been extensively tested and standardized by empirical research works. This paper describes the development of an automated method to evaluate a person's social skills level based on SRS-2. We use multimodal features, including BERT-based features, and perform score estimation with a 0.76 Pearson correlation coefficient while using feature selection. In addition, we examine the linguistic aspects of BERT-based features through subjective evaluations. Consequently, the BERT-based features show a strong negative correlation with human subjective scores of fluency, appropriate word choice, and understandable speech structure.

Publication: IEICE TRANSACTIONS on Information Vol.E105-D No.3 pp.578-586

Publication Date: 2022/03/01

Publicized: 2021/11/02

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2021HCP0009

Type of Manuscript: Special Section PAPER (Special Section on Human Communication IV)

Category

Authors

Takeshi SAGA
  Nara Institute of Science and Technology
Hiroki TANAKA
  Nara Institute of Science and Technology
Hidemi IWASAKA
  Nara Medical University
Satoshi NAKAMURA
  Nara Institute of Science and Technology

Keyword

social skills training, social communication, social responsiveness scale, linear regression, feature selection

Cite this

Copy

Takeshi SAGA, Hiroki TANAKA, Hidemi IWASAKA, Satoshi NAKAMURA, "Multimodal Prediction of Social Responsiveness Score with BERT-Based Text Features" in IEICE TRANSACTIONS on Information, vol. E105-D, no. 3, pp. 578-586, March 2022, doi: 10.1587/transinf.2021HCP0009.
Abstract: Social Skills Training (SST) has been used for years to improve individuals' social skills toward building a better daily life. In SST carried out by humans, the social skills level is usually evaluated through a verbal interview conducted by the trainer. Although this evaluation is based on psychiatric knowledge and professional experience, its quality depends on the trainer's capabilities. Therefore, to standardize such evaluations, quantifiable metrics are required. To meet this need, the second edition of the Social Responsiveness Scale (SRS-2) offers a viable solution because it has been extensively tested and standardized by empirical research works. This paper describes the development of an automated method to evaluate a person's social skills level based on SRS-2. We use multimodal features, including BERT-based features, and perform score estimation with a 0.76 Pearson correlation coefficient while using feature selection. In addition, we examine the linguistic aspects of BERT-based features through subjective evaluations. Consequently, the BERT-based features show a strong negative correlation with human subjective scores of fluency, appropriate word choice, and understandable speech structure.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2021HCP0009/_p

Copy

@ARTICLE{e105-d_3_578,
author={Takeshi SAGA, Hiroki TANAKA, Hidemi IWASAKA, Satoshi NAKAMURA, },
journal={IEICE TRANSACTIONS on Information},
title={Multimodal Prediction of Social Responsiveness Score with BERT-Based Text Features},
year={2022},
volume={E105-D},
number={3},
pages={578-586},
abstract={Social Skills Training (SST) has been used for years to improve individuals' social skills toward building a better daily life. In SST carried out by humans, the social skills level is usually evaluated through a verbal interview conducted by the trainer. Although this evaluation is based on psychiatric knowledge and professional experience, its quality depends on the trainer's capabilities. Therefore, to standardize such evaluations, quantifiable metrics are required. To meet this need, the second edition of the Social Responsiveness Scale (SRS-2) offers a viable solution because it has been extensively tested and standardized by empirical research works. This paper describes the development of an automated method to evaluate a person's social skills level based on SRS-2. We use multimodal features, including BERT-based features, and perform score estimation with a 0.76 Pearson correlation coefficient while using feature selection. In addition, we examine the linguistic aspects of BERT-based features through subjective evaluations. Consequently, the BERT-based features show a strong negative correlation with human subjective scores of fluency, appropriate word choice, and understandable speech structure.},
keywords={},
doi={10.1587/transinf.2021HCP0009},
ISSN={1745-1361},
month={March},}

Copy

TY - JOUR
TI - Multimodal Prediction of Social Responsiveness Score with BERT-Based Text Features
T2 - IEICE TRANSACTIONS on Information
SP - 578
EP - 586
AU - Takeshi SAGA
AU - Hiroki TANAKA
AU - Hidemi IWASAKA
AU - Satoshi NAKAMURA
PY - 2022
DO - 10.1587/transinf.2021HCP0009
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E105-D
IS - 3
JA - IEICE TRANSACTIONS on Information
Y1 - March 2022
AB - Social Skills Training (SST) has been used for years to improve individuals' social skills toward building a better daily life. In SST carried out by humans, the social skills level is usually evaluated through a verbal interview conducted by the trainer. Although this evaluation is based on psychiatric knowledge and professional experience, its quality depends on the trainer's capabilities. Therefore, to standardize such evaluations, quantifiable metrics are required. To meet this need, the second edition of the Social Responsiveness Scale (SRS-2) offers a viable solution because it has been extensively tested and standardized by empirical research works. This paper describes the development of an automated method to evaluate a person's social skills level based on SRS-2. We use multimodal features, including BERT-based features, and perform score estimation with a 0.76 Pearson correlation coefficient while using feature selection. In addition, we examine the linguistic aspects of BERT-based features through subjective evaluations. Consequently, the BERT-based features show a strong negative correlation with human subjective scores of fluency, appropriate word choice, and understandable speech structure.
ER -