IEICE TRANSACTIONS on Information
Pose-Free Face Swapping Based on a Deformable 3D Shape Morphable Model
Summary :
Traditional face swapping technologies require that the faces of source images and target images have similar pose and appearance (usually frontal). For overcoming this limit in applications this paper presents a pose-free face swapping method based on personalized 3D face modeling. By using a deformable 3D shape morphable model, a photo-realistic 3D face is reconstructed from a single frontal view image. With the aid of the generated 3D face, a virtual source image of the person with the same pose as the target face can be rendered, which is used as a source image for face swapping. To solve the problem of illumination difference between the target face and the source face, a color transfer merging method is proposed. It outperforms the original color transfer method in dealing with the illumination gap problem. An experiment shows that the proposed face reconstruction method is fast and efficient. In addition, we have conducted experiments of face swapping in a variety of scenarios such as children's story book, role play, and face de-identification stripping facial information used for identification, and promising results have been obtained.
- Publication
- IEICE TRANSACTIONS on Information Vol.E97-D No.2 pp.305-314
- Publication Date
- 2014/02/01
- Publicized
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.E97.D.305
- Type of Manuscript
- PAPER
- Category
- Computer Graphics
Authors
Yuan LIN
Tsinghua University
Shengjin WANG
Tsinghua University
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Yuan LIN, Shengjin WANG, "Pose-Free Face Swapping Based on a Deformable 3D Shape Morphable Model" in IEICE TRANSACTIONS on Information,
vol. E97-D, no. 2, pp. 305-314, February 2014, doi: 10.1587/transinf.E97.D.305.
Abstract: Traditional face swapping technologies require that the faces of source images and target images have similar pose and appearance (usually frontal). For overcoming this limit in applications this paper presents a pose-free face swapping method based on personalized 3D face modeling. By using a deformable 3D shape morphable model, a photo-realistic 3D face is reconstructed from a single frontal view image. With the aid of the generated 3D face, a virtual source image of the person with the same pose as the target face can be rendered, which is used as a source image for face swapping. To solve the problem of illumination difference between the target face and the source face, a color transfer merging method is proposed. It outperforms the original color transfer method in dealing with the illumination gap problem. An experiment shows that the proposed face reconstruction method is fast and efficient. In addition, we have conducted experiments of face swapping in a variety of scenarios such as children's story book, role play, and face de-identification stripping facial information used for identification, and promising results have been obtained.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.E97.D.305/_p
Copy
@ARTICLE{e97-d_2_305,
author={Yuan LIN, Shengjin WANG, },
journal={IEICE TRANSACTIONS on Information},
title={Pose-Free Face Swapping Based on a Deformable 3D Shape Morphable Model},
year={2014},
volume={E97-D},
number={2},
pages={305-314},
abstract={Traditional face swapping technologies require that the faces of source images and target images have similar pose and appearance (usually frontal). For overcoming this limit in applications this paper presents a pose-free face swapping method based on personalized 3D face modeling. By using a deformable 3D shape morphable model, a photo-realistic 3D face is reconstructed from a single frontal view image. With the aid of the generated 3D face, a virtual source image of the person with the same pose as the target face can be rendered, which is used as a source image for face swapping. To solve the problem of illumination difference between the target face and the source face, a color transfer merging method is proposed. It outperforms the original color transfer method in dealing with the illumination gap problem. An experiment shows that the proposed face reconstruction method is fast and efficient. In addition, we have conducted experiments of face swapping in a variety of scenarios such as children's story book, role play, and face de-identification stripping facial information used for identification, and promising results have been obtained.},
keywords={},
doi={10.1587/transinf.E97.D.305},
ISSN={1745-1361},
month={February},}
Copy
TY - JOUR
TI - Pose-Free Face Swapping Based on a Deformable 3D Shape Morphable Model
T2 - IEICE TRANSACTIONS on Information
SP - 305
EP - 314
AU - Yuan LIN
AU - Shengjin WANG
PY - 2014
DO - 10.1587/transinf.E97.D.305
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E97-D
IS - 2
JA - IEICE TRANSACTIONS on Information
Y1 - February 2014
AB - Traditional face swapping technologies require that the faces of source images and target images have similar pose and appearance (usually frontal). For overcoming this limit in applications this paper presents a pose-free face swapping method based on personalized 3D face modeling. By using a deformable 3D shape morphable model, a photo-realistic 3D face is reconstructed from a single frontal view image. With the aid of the generated 3D face, a virtual source image of the person with the same pose as the target face can be rendered, which is used as a source image for face swapping. To solve the problem of illumination difference between the target face and the source face, a color transfer merging method is proposed. It outperforms the original color transfer method in dealing with the illumination gap problem. An experiment shows that the proposed face reconstruction method is fast and efficient. In addition, we have conducted experiments of face swapping in a variety of scenarios such as children's story book, role play, and face de-identification stripping facial information used for identification, and promising results have been obtained.
ER -
English
