GazeFollowTR: A Method of Gaze Following with Reborn Mechanism
Summary :
Gaze following is the task of estimating where an observer is looking inside a scene. Both the observer and scene information must be learned to determine the gaze directions and gaze points. Recently, many existing works have only focused on scenes or observers. In contrast, revealed frameworks for gaze following are limited. In this paper, a gaze following method using a hybrid transformer is proposed. Based on the conventional method (GazeFollow), we conduct three developments. First, a hybrid transformer is applied for learning head images and gaze positions. Second, the pinball loss function is utilized to control the gaze point error. Finally, a novel ReLU layer with the reborn mechanism (reborn ReLU) is conducted to replace traditional ReLU layers in different network stages. To test the performance of our developments, we train our developed framework with the DL Gaze dataset and evaluate the model on our collected set. Through our experimental results, it can be proven that our framework can achieve outperformance over our referred methods.
- Publication
- IEICE TRANSACTIONS on Fundamentals Vol.E106-A No.6 pp.938-946
- Publication Date
- 2023/06/01
- Publicized
- 2022/11/30
- Online ISSN
- 1745-1337
- DOI
- 10.1587/transfun.2022EAP1068
- Type of Manuscript
- PAPER
- Category
- Vision
Authors
Jingzhao DAI
Nanjing University
Ming LI
Nanjing University
Xuejiao HU
Nanjing University
Yang LI
Nanjing University
Sidan DU
Nanjing University
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Jingzhao DAI, Ming LI, Xuejiao HU, Yang LI, Sidan DU, "GazeFollowTR: A Method of Gaze Following with Reborn Mechanism" in IEICE TRANSACTIONS on Fundamentals,
vol. E106-A, no. 6, pp. 938-946, June 2023, doi: 10.1587/transfun.2022EAP1068.
Abstract: Gaze following is the task of estimating where an observer is looking inside a scene. Both the observer and scene information must be learned to determine the gaze directions and gaze points. Recently, many existing works have only focused on scenes or observers. In contrast, revealed frameworks for gaze following are limited. In this paper, a gaze following method using a hybrid transformer is proposed. Based on the conventional method (GazeFollow), we conduct three developments. First, a hybrid transformer is applied for learning head images and gaze positions. Second, the pinball loss function is utilized to control the gaze point error. Finally, a novel ReLU layer with the reborn mechanism (reborn ReLU) is conducted to replace traditional ReLU layers in different network stages. To test the performance of our developments, we train our developed framework with the DL Gaze dataset and evaluate the model on our collected set. Through our experimental results, it can be proven that our framework can achieve outperformance over our referred methods.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.2022EAP1068/_p
Copy
@ARTICLE{e106-a_6_938,
author={Jingzhao DAI, Ming LI, Xuejiao HU, Yang LI, Sidan DU, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={GazeFollowTR: A Method of Gaze Following with Reborn Mechanism},
year={2023},
volume={E106-A},
number={6},
pages={938-946},
abstract={Gaze following is the task of estimating where an observer is looking inside a scene. Both the observer and scene information must be learned to determine the gaze directions and gaze points. Recently, many existing works have only focused on scenes or observers. In contrast, revealed frameworks for gaze following are limited. In this paper, a gaze following method using a hybrid transformer is proposed. Based on the conventional method (GazeFollow), we conduct three developments. First, a hybrid transformer is applied for learning head images and gaze positions. Second, the pinball loss function is utilized to control the gaze point error. Finally, a novel ReLU layer with the reborn mechanism (reborn ReLU) is conducted to replace traditional ReLU layers in different network stages. To test the performance of our developments, we train our developed framework with the DL Gaze dataset and evaluate the model on our collected set. Through our experimental results, it can be proven that our framework can achieve outperformance over our referred methods.},
keywords={},
doi={10.1587/transfun.2022EAP1068},
ISSN={1745-1337},
month={June},}
Copy
TY - JOUR
TI - GazeFollowTR: A Method of Gaze Following with Reborn Mechanism
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 938
EP - 946
AU - Jingzhao DAI
AU - Ming LI
AU - Xuejiao HU
AU - Yang LI
AU - Sidan DU
PY - 2023
DO - 10.1587/transfun.2022EAP1068
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E106-A
IS - 6
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - June 2023
AB - Gaze following is the task of estimating where an observer is looking inside a scene. Both the observer and scene information must be learned to determine the gaze directions and gaze points. Recently, many existing works have only focused on scenes or observers. In contrast, revealed frameworks for gaze following are limited. In this paper, a gaze following method using a hybrid transformer is proposed. Based on the conventional method (GazeFollow), we conduct three developments. First, a hybrid transformer is applied for learning head images and gaze positions. Second, the pinball loss function is utilized to control the gaze point error. Finally, a novel ReLU layer with the reborn mechanism (reborn ReLU) is conducted to replace traditional ReLU layers in different network stages. To test the performance of our developments, we train our developed framework with the DL Gaze dataset and evaluate the model on our collected set. Through our experimental results, it can be proven that our framework can achieve outperformance over our referred methods.
ER -
English
