Driver Drowsiness Estimation by Parallel Linked Time-Domain CNN with Novel Temporal Measures on Eye States

Kenta NISHIYUKI; Jia-Yau SHIAU; Shigenori NAGAE; Tomohiro YABUUCHI; Koichi KINOSHITA; Yuki HASEGAWA; Takayoshi YAMASHITA; Hironobu FUJIYOSHI

doi:10.1587/transinf.2019MVP0017

Driver Drowsiness Estimation by Parallel Linked Time-Domain CNN with Novel Temporal Measures on Eye States

Kenta NISHIYUKI, Jia-Yau SHIAU, Shigenori NAGAE, Tomohiro YABUUCHI, Koichi KINOSHITA, Yuki HASEGAWA, Takayoshi YAMASHITA, Hironobu FUJIYOSHI

Full Text Views

0

Cite this

Summary :

Driver drowsiness estimation is one of the important tasks for preventing car accidents. Most of the approaches are binary classification that classify a driver is significantly drowsy or not. Multi-level drowsiness estimation, that detects not only significant drowsiness but also moderate drowsiness, is helpful to a safer and more comfortable car system. Existing approaches are mostly based on conventional temporal measures which extract temporal information related to eye states, and these measures mainly focus on detecting significant drowsiness for binary classification. For multi-level drowsiness estimation, we propose two temporal measures, average eye closed time (AECT) and soft percentage of eyelid closure (Soft PERCLOS). Existing approaches are also based on a time domain convolutional neural network (CNN) as deep neural network models, of which layers are linked sequentially. The network model extracts features mainly focusing on mono-temporal resolution. We found that features focusing on multi-temporal resolution are effective to multi-level drowsiness estimation, and we propose a parallel linked time-domain CNN to extract the multi-temporal features. We collected an own dataset in a real environment and evaluated the proposed methods with the dataset. Compared with existing temporal measures and network models, Our system outperforms the existing approaches on the dataset.

Publication: IEICE TRANSACTIONS on Information Vol.E103-D No.6 pp.1276-1286

Publication Date: 2020/06/01

Publicized: 2020/04/10

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2019MVP0017

Type of Manuscript: Special Section PAPER (Special Section on Machine Vision and its Applications)

Category

Authors

Kenta NISHIYUKI
  OMRON Corporation,Chubu University
Jia-Yau SHIAU
  National Taiwan University
Shigenori NAGAE
  OMRON Corporation
Tomohiro YABUUCHI
  OMRON Corporation
Koichi KINOSHITA
  OMRON Corporation
Yuki HASEGAWA
  OMRON Corporation
Takayoshi YAMASHITA
  Chubu University
Hironobu FUJIYOSHI
  Chubu University

Keyword

driver monitoring, driver drowsiness estimation, time-domain CNN, PERCLOS

Cite this

Copy

Kenta NISHIYUKI, Jia-Yau SHIAU, Shigenori NAGAE, Tomohiro YABUUCHI, Koichi KINOSHITA, Yuki HASEGAWA, Takayoshi YAMASHITA, Hironobu FUJIYOSHI, "Driver Drowsiness Estimation by Parallel Linked Time-Domain CNN with Novel Temporal Measures on Eye States" in IEICE TRANSACTIONS on Information, vol. E103-D, no. 6, pp. 1276-1286, June 2020, doi: 10.1587/transinf.2019MVP0017.
Abstract: Driver drowsiness estimation is one of the important tasks for preventing car accidents. Most of the approaches are binary classification that classify a driver is significantly drowsy or not. Multi-level drowsiness estimation, that detects not only significant drowsiness but also moderate drowsiness, is helpful to a safer and more comfortable car system. Existing approaches are mostly based on conventional temporal measures which extract temporal information related to eye states, and these measures mainly focus on detecting significant drowsiness for binary classification. For multi-level drowsiness estimation, we propose two temporal measures, average eye closed time (AECT) and soft percentage of eyelid closure (Soft PERCLOS). Existing approaches are also based on a time domain convolutional neural network (CNN) as deep neural network models, of which layers are linked sequentially. The network model extracts features mainly focusing on mono-temporal resolution. We found that features focusing on multi-temporal resolution are effective to multi-level drowsiness estimation, and we propose a parallel linked time-domain CNN to extract the multi-temporal features. We collected an own dataset in a real environment and evaluated the proposed methods with the dataset. Compared with existing temporal measures and network models, Our system outperforms the existing approaches on the dataset.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019MVP0017/_p

Copy

@ARTICLE{e103-d_6_1276,
author={Kenta NISHIYUKI, Jia-Yau SHIAU, Shigenori NAGAE, Tomohiro YABUUCHI, Koichi KINOSHITA, Yuki HASEGAWA, Takayoshi YAMASHITA, Hironobu FUJIYOSHI, },
journal={IEICE TRANSACTIONS on Information},
title={Driver Drowsiness Estimation by Parallel Linked Time-Domain CNN with Novel Temporal Measures on Eye States},
year={2020},
volume={E103-D},
number={6},
pages={1276-1286},
abstract={Driver drowsiness estimation is one of the important tasks for preventing car accidents. Most of the approaches are binary classification that classify a driver is significantly drowsy or not. Multi-level drowsiness estimation, that detects not only significant drowsiness but also moderate drowsiness, is helpful to a safer and more comfortable car system. Existing approaches are mostly based on conventional temporal measures which extract temporal information related to eye states, and these measures mainly focus on detecting significant drowsiness for binary classification. For multi-level drowsiness estimation, we propose two temporal measures, average eye closed time (AECT) and soft percentage of eyelid closure (Soft PERCLOS). Existing approaches are also based on a time domain convolutional neural network (CNN) as deep neural network models, of which layers are linked sequentially. The network model extracts features mainly focusing on mono-temporal resolution. We found that features focusing on multi-temporal resolution are effective to multi-level drowsiness estimation, and we propose a parallel linked time-domain CNN to extract the multi-temporal features. We collected an own dataset in a real environment and evaluated the proposed methods with the dataset. Compared with existing temporal measures and network models, Our system outperforms the existing approaches on the dataset.},
keywords={},
doi={10.1587/transinf.2019MVP0017},
ISSN={1745-1361},
month={June},}

Copy

TY - JOUR
TI - Driver Drowsiness Estimation by Parallel Linked Time-Domain CNN with Novel Temporal Measures on Eye States
T2 - IEICE TRANSACTIONS on Information
SP - 1276
EP - 1286
AU - Kenta NISHIYUKI
AU - Jia-Yau SHIAU
AU - Shigenori NAGAE
AU - Tomohiro YABUUCHI
AU - Koichi KINOSHITA
AU - Yuki HASEGAWA
AU - Takayoshi YAMASHITA
AU - Hironobu FUJIYOSHI
PY - 2020
DO - 10.1587/transinf.2019MVP0017
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E103-D
IS - 6
JA - IEICE TRANSACTIONS on Information
Y1 - June 2020
AB - Driver drowsiness estimation is one of the important tasks for preventing car accidents. Most of the approaches are binary classification that classify a driver is significantly drowsy or not. Multi-level drowsiness estimation, that detects not only significant drowsiness but also moderate drowsiness, is helpful to a safer and more comfortable car system. Existing approaches are mostly based on conventional temporal measures which extract temporal information related to eye states, and these measures mainly focus on detecting significant drowsiness for binary classification. For multi-level drowsiness estimation, we propose two temporal measures, average eye closed time (AECT) and soft percentage of eyelid closure (Soft PERCLOS). Existing approaches are also based on a time domain convolutional neural network (CNN) as deep neural network models, of which layers are linked sequentially. The network model extracts features mainly focusing on mono-temporal resolution. We found that features focusing on multi-temporal resolution are effective to multi-level drowsiness estimation, and we propose a parallel linked time-domain CNN to extract the multi-temporal features. We collected an own dataset in a real environment and evaluated the proposed methods with the dataset. Compared with existing temporal measures and network models, Our system outperforms the existing approaches on the dataset.
ER -