IEICE TRANSACTIONS on Fundamentals
Heterogeneous-Graph-Based Video Search Reranking Using Topic Relevance
Summary :
In this paper, we address the problem of analyzing topics, included in a social video group, to improve the retrieval performance of videos. Unlike previous methods that focused on an individual visual aspect of videos, the proposed method aims to leverage the “mutual reinforcement” of heterogeneous modalities such as tags and users associated with video on the Internet. To represent multiple types of relationships between each heterogeneous modality, the proposed method constructs three subgraphs: user-tag, video-video, and video-tag graphs. We combine the three types of graphs to obtain a heterogeneous graph. Then the extraction of latent features, i.e., topics, becomes feasible by applying graph-based soft clustering to the heterogeneous graph. By estimating the membership of each grouped cluster for each video, the proposed method defines a new video similarity measure. Since the understanding of video content is enhanced by exploiting latent features obtained from different types of data that complement each other, the performance of visual reranking is improved by the proposed method. Results of experiments on a video dataset that consists of YouTube-8M videos show the effectiveness of the proposed method, which achieves a 24.3% improvement in terms of the mean normalized discounted cumulative gain in a search ranking task compared with the baseline method.
- Publication
- IEICE TRANSACTIONS on Fundamentals Vol.E103-A No.12 pp.1529-1540
- Publication Date
- 2020/12/01
- Publicized
- Online ISSN
- 1745-1337
- DOI
- 10.1587/transfun.2020SMP0023
- Type of Manuscript
- Special Section PAPER (Special Section on Smart Multimedia & Communication Systems)
- Category
- Vision
Authors
Soh YOSHIDA
Kansai University
Mitsuji MUNEYASU
Kansai University
Takahiro OGAWA
Hokkaido University
Miki HASEYAMA
Hokkaido University
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Soh YOSHIDA, Mitsuji MUNEYASU, Takahiro OGAWA, Miki HASEYAMA, "Heterogeneous-Graph-Based Video Search Reranking Using Topic Relevance" in IEICE TRANSACTIONS on Fundamentals,
vol. E103-A, no. 12, pp. 1529-1540, December 2020, doi: 10.1587/transfun.2020SMP0023.
Abstract: In this paper, we address the problem of analyzing topics, included in a social video group, to improve the retrieval performance of videos. Unlike previous methods that focused on an individual visual aspect of videos, the proposed method aims to leverage the “mutual reinforcement” of heterogeneous modalities such as tags and users associated with video on the Internet. To represent multiple types of relationships between each heterogeneous modality, the proposed method constructs three subgraphs: user-tag, video-video, and video-tag graphs. We combine the three types of graphs to obtain a heterogeneous graph. Then the extraction of latent features, i.e., topics, becomes feasible by applying graph-based soft clustering to the heterogeneous graph. By estimating the membership of each grouped cluster for each video, the proposed method defines a new video similarity measure. Since the understanding of video content is enhanced by exploiting latent features obtained from different types of data that complement each other, the performance of visual reranking is improved by the proposed method. Results of experiments on a video dataset that consists of YouTube-8M videos show the effectiveness of the proposed method, which achieves a 24.3% improvement in terms of the mean normalized discounted cumulative gain in a search ranking task compared with the baseline method.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.2020SMP0023/_p
Copy
@ARTICLE{e103-a_12_1529,
author={Soh YOSHIDA, Mitsuji MUNEYASU, Takahiro OGAWA, Miki HASEYAMA, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Heterogeneous-Graph-Based Video Search Reranking Using Topic Relevance},
year={2020},
volume={E103-A},
number={12},
pages={1529-1540},
abstract={In this paper, we address the problem of analyzing topics, included in a social video group, to improve the retrieval performance of videos. Unlike previous methods that focused on an individual visual aspect of videos, the proposed method aims to leverage the “mutual reinforcement” of heterogeneous modalities such as tags and users associated with video on the Internet. To represent multiple types of relationships between each heterogeneous modality, the proposed method constructs three subgraphs: user-tag, video-video, and video-tag graphs. We combine the three types of graphs to obtain a heterogeneous graph. Then the extraction of latent features, i.e., topics, becomes feasible by applying graph-based soft clustering to the heterogeneous graph. By estimating the membership of each grouped cluster for each video, the proposed method defines a new video similarity measure. Since the understanding of video content is enhanced by exploiting latent features obtained from different types of data that complement each other, the performance of visual reranking is improved by the proposed method. Results of experiments on a video dataset that consists of YouTube-8M videos show the effectiveness of the proposed method, which achieves a 24.3% improvement in terms of the mean normalized discounted cumulative gain in a search ranking task compared with the baseline method.},
keywords={},
doi={10.1587/transfun.2020SMP0023},
ISSN={1745-1337},
month={December},}
Copy
TY - JOUR
TI - Heterogeneous-Graph-Based Video Search Reranking Using Topic Relevance
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 1529
EP - 1540
AU - Soh YOSHIDA
AU - Mitsuji MUNEYASU
AU - Takahiro OGAWA
AU - Miki HASEYAMA
PY - 2020
DO - 10.1587/transfun.2020SMP0023
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E103-A
IS - 12
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - December 2020
AB - In this paper, we address the problem of analyzing topics, included in a social video group, to improve the retrieval performance of videos. Unlike previous methods that focused on an individual visual aspect of videos, the proposed method aims to leverage the “mutual reinforcement” of heterogeneous modalities such as tags and users associated with video on the Internet. To represent multiple types of relationships between each heterogeneous modality, the proposed method constructs three subgraphs: user-tag, video-video, and video-tag graphs. We combine the three types of graphs to obtain a heterogeneous graph. Then the extraction of latent features, i.e., topics, becomes feasible by applying graph-based soft clustering to the heterogeneous graph. By estimating the membership of each grouped cluster for each video, the proposed method defines a new video similarity measure. Since the understanding of video content is enhanced by exploiting latent features obtained from different types of data that complement each other, the performance of visual reranking is improved by the proposed method. Results of experiments on a video dataset that consists of YouTube-8M videos show the effectiveness of the proposed method, which achieves a 24.3% improvement in terms of the mean normalized discounted cumulative gain in a search ranking task compared with the baseline method.
ER -
English
