Distributed Video Decoding on Hadoop
Summary :
Video analytics is usually time-consuming as it not only requires video decoding as a first step but also usually applies complex computer vision and machine learning algorithms to the decoded frame. To achieve high efficiency in video analytics with ever increasing frame size, many researches have been conducted for distributed video processing using Hadoop. However, most approaches focused on processing multiple video files on multiple nodes. Such approaches require a number of video files to achieve any speedup, and could easily result in load imbalance when the size of video files is reasonably long since a video file itself is processed sequentially. In contrast, we propose a distributed video decoding method with an extended FFmpeg and VideoRecordReader, by which a single large video file can be processed in parallel across multiple nodes in Hadoop. The experimental results show that a case study of face detection and SURF system achieve 40.6 times and 29.1 times of speedups respectively on a four-node cluster with 12 mappers in each node, showing good scalability.
- Publication
- IEICE TRANSACTIONS on Information Vol.E101-D No.12 pp.2933-2941
- Publication Date
- 2018/12/01
- Publicized
- 2018/09/18
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.2018PAP0014
- Type of Manuscript
- Special Section PAPER (Special Section on Parallel and Distributed Computing and Networking)
- Category
- Cluster Computing
Authors
Illo YOON
University of Seoul
Saehanseul YI
University of Seoul
Chanyoung OH
University of Seoul
Hyeonjin JUNG
University of Seoul
Youngmin YI
University of Seoul
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Illo YOON, Saehanseul YI, Chanyoung OH, Hyeonjin JUNG, Youngmin YI, "Distributed Video Decoding on Hadoop" in IEICE TRANSACTIONS on Information,
vol. E101-D, no. 12, pp. 2933-2941, December 2018, doi: 10.1587/transinf.2018PAP0014.
Abstract: Video analytics is usually time-consuming as it not only requires video decoding as a first step but also usually applies complex computer vision and machine learning algorithms to the decoded frame. To achieve high efficiency in video analytics with ever increasing frame size, many researches have been conducted for distributed video processing using Hadoop. However, most approaches focused on processing multiple video files on multiple nodes. Such approaches require a number of video files to achieve any speedup, and could easily result in load imbalance when the size of video files is reasonably long since a video file itself is processed sequentially. In contrast, we propose a distributed video decoding method with an extended FFmpeg and VideoRecordReader, by which a single large video file can be processed in parallel across multiple nodes in Hadoop. The experimental results show that a case study of face detection and SURF system achieve 40.6 times and 29.1 times of speedups respectively on a four-node cluster with 12 mappers in each node, showing good scalability.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2018PAP0014/_p
Copy
@ARTICLE{e101-d_12_2933,
author={Illo YOON, Saehanseul YI, Chanyoung OH, Hyeonjin JUNG, Youngmin YI, },
journal={IEICE TRANSACTIONS on Information},
title={Distributed Video Decoding on Hadoop},
year={2018},
volume={E101-D},
number={12},
pages={2933-2941},
abstract={Video analytics is usually time-consuming as it not only requires video decoding as a first step but also usually applies complex computer vision and machine learning algorithms to the decoded frame. To achieve high efficiency in video analytics with ever increasing frame size, many researches have been conducted for distributed video processing using Hadoop. However, most approaches focused on processing multiple video files on multiple nodes. Such approaches require a number of video files to achieve any speedup, and could easily result in load imbalance when the size of video files is reasonably long since a video file itself is processed sequentially. In contrast, we propose a distributed video decoding method with an extended FFmpeg and VideoRecordReader, by which a single large video file can be processed in parallel across multiple nodes in Hadoop. The experimental results show that a case study of face detection and SURF system achieve 40.6 times and 29.1 times of speedups respectively on a four-node cluster with 12 mappers in each node, showing good scalability.},
keywords={},
doi={10.1587/transinf.2018PAP0014},
ISSN={1745-1361},
month={December},}
Copy
TY - JOUR
TI - Distributed Video Decoding on Hadoop
T2 - IEICE TRANSACTIONS on Information
SP - 2933
EP - 2941
AU - Illo YOON
AU - Saehanseul YI
AU - Chanyoung OH
AU - Hyeonjin JUNG
AU - Youngmin YI
PY - 2018
DO - 10.1587/transinf.2018PAP0014
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E101-D
IS - 12
JA - IEICE TRANSACTIONS on Information
Y1 - December 2018
AB - Video analytics is usually time-consuming as it not only requires video decoding as a first step but also usually applies complex computer vision and machine learning algorithms to the decoded frame. To achieve high efficiency in video analytics with ever increasing frame size, many researches have been conducted for distributed video processing using Hadoop. However, most approaches focused on processing multiple video files on multiple nodes. Such approaches require a number of video files to achieve any speedup, and could easily result in load imbalance when the size of video files is reasonably long since a video file itself is processed sequentially. In contrast, we propose a distributed video decoding method with an extended FFmpeg and VideoRecordReader, by which a single large video file can be processed in parallel across multiple nodes in Hadoop. The experimental results show that a case study of face detection and SURF system achieve 40.6 times and 29.1 times of speedups respectively on a four-node cluster with 12 mappers in each node, showing good scalability.
ER -
English
