Development of an Estimation Model for Instantaneous Presence in Audio-Visual Content

Kenji OZAWA; Shota TSUKAHARA; Yuichiro KINOSHITA; Masanori MORISE

doi:10.1587/transinf.2015MUP0014

IEICE TRANSACTIONS on Information

Development of an Estimation Model for Instantaneous Presence in Audio-Visual Content

Kenji OZAWA, Shota TSUKAHARA, Yuichiro KINOSHITA, Masanori MORISE

Full Text Views

0

Cite this

Summary :

The sense of presence is often used to evaluate the performances of audio-visual (AV) content and systems. However, a presence meter has yet to be realized. We consider that the sense of presence can be divided into two aspects: system presence and content presence. In this study we focused on content presence. To estimate the overall presence of a content item, we have developed estimation models for the sense of presence in audio-only and audio-visual content. In this study, the audio-visual model is expanded to estimate the instantaneous presence in an AV content item. Initially, we conducted an evaluation experiment of the presence with 40 content items to investigate the relationship between the features of the AV content and the instantaneous presence. Based on the experimental data, a neural-network-based model was developed by expanding the previous model. To express the variation in instantaneous presence, 6 audio-related features and 14 visual-related features, which are extracted from the content items in 500-ms intervals, are used as inputs for the model. The audio-related features are loudness, sharpness, roughness, dynamic range and standard deviation in sound pressure levels, and movement of sound images. The visual-related features involve hue, lightness, saturation, and movement of visual images. After constructing the model, a generalization test confirmed that the model is sufficiently accurate to estimate the instantaneous presence. Hence, the model should contribute to the development of a presence meter.

Publication: IEICE TRANSACTIONS on Information Vol.E99-D No.1 pp.120-127

Publication Date: 2016/01/01

Publicized: 2015/10/21

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2015MUP0014

Type of Manuscript: Special Section PAPER (Special Section on Enriched Multimedia---Creation of a New Society through Value-added Multimedia Content---)

Category

Authors

Kenji OZAWA
  University of Yamanashi
Shota TSUKAHARA
  University of Yamanashi
Yuichiro KINOSHITA
  University of Yamanashi
Masanori MORISE
  University of Yamanashi

Keyword

sense of presence, instantaneous presence, audio-visual content, neural networks, time-series data

Cite this

Copy

Kenji OZAWA, Shota TSUKAHARA, Yuichiro KINOSHITA, Masanori MORISE, "Development of an Estimation Model for Instantaneous Presence in Audio-Visual Content" in IEICE TRANSACTIONS on Information, vol. E99-D, no. 1, pp. 120-127, January 2016, doi: 10.1587/transinf.2015MUP0014.
Abstract: The sense of presence is often used to evaluate the performances of audio-visual (AV) content and systems. However, a presence meter has yet to be realized. We consider that the sense of presence can be divided into two aspects: system presence and content presence. In this study we focused on content presence. To estimate the overall presence of a content item, we have developed estimation models for the sense of presence in audio-only and audio-visual content. In this study, the audio-visual model is expanded to estimate the instantaneous presence in an AV content item. Initially, we conducted an evaluation experiment of the presence with 40 content items to investigate the relationship between the features of the AV content and the instantaneous presence. Based on the experimental data, a neural-network-based model was developed by expanding the previous model. To express the variation in instantaneous presence, 6 audio-related features and 14 visual-related features, which are extracted from the content items in 500-ms intervals, are used as inputs for the model. The audio-related features are loudness, sharpness, roughness, dynamic range and standard deviation in sound pressure levels, and movement of sound images. The visual-related features involve hue, lightness, saturation, and movement of visual images. After constructing the model, a generalization test confirmed that the model is sufficiently accurate to estimate the instantaneous presence. Hence, the model should contribute to the development of a presence meter.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2015MUP0014/_p

Copy

@ARTICLE{e99-d_1_120,
author={Kenji OZAWA, Shota TSUKAHARA, Yuichiro KINOSHITA, Masanori MORISE, },
journal={IEICE TRANSACTIONS on Information},
title={Development of an Estimation Model for Instantaneous Presence in Audio-Visual Content},
year={2016},
volume={E99-D},
number={1},
pages={120-127},
abstract={The sense of presence is often used to evaluate the performances of audio-visual (AV) content and systems. However, a presence meter has yet to be realized. We consider that the sense of presence can be divided into two aspects: system presence and content presence. In this study we focused on content presence. To estimate the overall presence of a content item, we have developed estimation models for the sense of presence in audio-only and audio-visual content. In this study, the audio-visual model is expanded to estimate the instantaneous presence in an AV content item. Initially, we conducted an evaluation experiment of the presence with 40 content items to investigate the relationship between the features of the AV content and the instantaneous presence. Based on the experimental data, a neural-network-based model was developed by expanding the previous model. To express the variation in instantaneous presence, 6 audio-related features and 14 visual-related features, which are extracted from the content items in 500-ms intervals, are used as inputs for the model. The audio-related features are loudness, sharpness, roughness, dynamic range and standard deviation in sound pressure levels, and movement of sound images. The visual-related features involve hue, lightness, saturation, and movement of visual images. After constructing the model, a generalization test confirmed that the model is sufficiently accurate to estimate the instantaneous presence. Hence, the model should contribute to the development of a presence meter.},
keywords={},
doi={10.1587/transinf.2015MUP0014},
ISSN={1745-1361},
month={January},}

Copy

TY - JOUR
TI - Development of an Estimation Model for Instantaneous Presence in Audio-Visual Content
T2 - IEICE TRANSACTIONS on Information
SP - 120
EP - 127
AU - Kenji OZAWA
AU - Shota TSUKAHARA
AU - Yuichiro KINOSHITA
AU - Masanori MORISE
PY - 2016
DO - 10.1587/transinf.2015MUP0014
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E99-D
IS - 1
JA - IEICE TRANSACTIONS on Information
Y1 - January 2016
AB - The sense of presence is often used to evaluate the performances of audio-visual (AV) content and systems. However, a presence meter has yet to be realized. We consider that the sense of presence can be divided into two aspects: system presence and content presence. In this study we focused on content presence. To estimate the overall presence of a content item, we have developed estimation models for the sense of presence in audio-only and audio-visual content. In this study, the audio-visual model is expanded to estimate the instantaneous presence in an AV content item. Initially, we conducted an evaluation experiment of the presence with 40 content items to investigate the relationship between the features of the AV content and the instantaneous presence. Based on the experimental data, a neural-network-based model was developed by expanding the previous model. To express the variation in instantaneous presence, 6 audio-related features and 14 visual-related features, which are extracted from the content items in 500-ms intervals, are used as inputs for the model. The audio-related features are loudness, sharpness, roughness, dynamic range and standard deviation in sound pressure levels, and movement of sound images. The visual-related features involve hue, lightness, saturation, and movement of visual images. After constructing the model, a generalization test confirmed that the model is sufficiently accurate to estimate the instantaneous presence. Hence, the model should contribute to the development of a presence meter.
ER -

IEICE TRANSACTIONS on Information