There is a strange trick to remove double chin, Zhejiang University post-00s undergraduates’ brand new beauty algorithm boarded on ACM SIGGRAPH

This method can successfully remove the double chin of the input portrait image while keeping other features unchanged.

Researchers from the State Key Laboratory of Computer-Aided Design and Graphics at Zhejiang University and the Zhejiang University-Tencent Game Intelligent Graphics Innovation Technology Joint Laboratory have proposed a method of training the fine separation boundaries in the hidden space of StyleGAN, using only one vector. Edit semantic features while keeping other facial features unchanged. This method is effective in removing double chin and other applications.

With the popularity of social networks, live broadcasts, and short videos, in order to leave a better impression on others, the application range of face editing “beauty” has become more and more extensive, and the continuous development of science and technology has resulted in a lot of face editing. Research branch. Among them, the hidden space of Generative Adversarial Networks (GAN) has always been a hot issue. Now more and more work focuses on the manipulation of hidden codes and semantic decoupling in hidden spaces. StyleGAN is a generative confrontation network that can generate high-quality face images, and its hidden space has very good linear characteristics. This feature of StyleGAN can be used to achieve high-quality face editing with a wide range of application scenarios. However, how to change specific features while keeping other irrelevant features unchanged, that is, to decouple features, is still a difficult problem. 

To solve this problem, researchers from the State Key Laboratory of Computer-Aided Design and Graphics of Zhejiang University and the Zhejiang University-Tencent Game Intelligent Graphics Innovation Technology Joint Laboratory have proposed a method to train the fine separation boundaries in the hidden space of StyleGAN. One vector can be used to edit semantic features while keeping other facial features unchanged . 

Taking double chin removal as an example, this method has a significant effect: 

There is a strange trick to remove double chin, Zhejiang University post-00s undergraduates' brand new beauty algorithm boarded on ACM SIGGRAPH

Figure 1: Portrait image with double chin (first row), new portrait with double chin removed (second row). 

The research paper “Coarse-to-Fine: Facial Structure Editing of Portrait Images via Latent Space Classifications” has been accepted by ACM SIGGRAPH 2021, the top international academic conference of computer graphics. 

There is a strange trick to remove double chin, Zhejiang University post-00s undergraduates' brand new beauty algorithm boarded on ACM SIGGRAPH

Paper address: http://www.cad.zju.edu.cn/home/jin/sig2021/sig2021.htm 

research direction

In the CV field, generating the hidden space of the confrontation network has always been a hot issue, and now more and more work is focusing on the manipulation of hidden codes. InterFaceGAN explores how the hidden space of the generated confrontation network is encoded, and proposes a method of editing semantic attributes using a separation boundary; In-domain GAN can invert the input image into the hidden space of the generated confrontation network, And as a regularizer to fine-tune the hidden code, and propose a method of semantic diffusion. 

In view of the importance of hidden space for StyleGAN research, more and more work has begun to focus on how to efficiently and high-quality invert the image back to the hidden space of StyleGAN, and get the corresponding hidden code; on this basis, based on StyleGAN The projector can directly invert the image back to the hidden space, thereby performing image-to-image conversion, realizing face pose changes, linear interpolation between faces, etc., Image2StyleGAN can invert the image back to the hidden space and perform semantic editing . 

Combining hidden codes and 3D models can also be used to parametrically adjust face features. GIF applies StyleGAN to a generated 3D face model (FLAME) to explicitly control the generated images; StyleRig uses StyleGAN and 3DMM for facial features Binding control, parameterized adjustment of the face. 

Research ideas

The core idea of ​​the new research is to train the fine separation boundary in the hidden space of StyleGAN. The separation boundary is a hyperplane in the hidden space proposed by InterFaceGAN, but the separation boundary trained by InterFaceGAN cannot separate irrelevant features. This paper proposes a well-designed training process to generate pairs of hidden codes with only specific features changed (in the example of removing double chins, these hidden codes basically keep the same except for the double chin). From these pairs of hidden codes The fine separation boundary is trained in the code, so as to realize the facial structure editing. 

The research first trains a double chin classifier. According to the presence or absence of double chins, the cryptic code in StyleGAN’s hidden space is scored, and then randomly sampled cryptic codes and their corresponding chin scores are used for training to obtain a rough separation. Border, used to synthesize a middle portrait without a double chin. In this process, other facial features, such as face shape and pose, cannot be preserved well after being edited by the rough separation boundary. 

To solve this problem, the study introduced a semantic diffusion method using a mask neck separated from other features, characteristics can double chin, the intermediary new semantic diffusion chin image to the original image, thereby obtaining A portrait image with no double chin and facial features and its corresponding hidden code. Finally, the study used pairs of cryptic codes with and without double chins to train a fine double chin separation boundary. 

In the test phase, the input hidden code is edited using the fine double chin separation boundary, and the image deformation algorithm is used to optimize the slight misalignment of the input and output images at the edge of the face to obtain the final result. 

There is a strange trick to remove double chin, Zhejiang University post-00s undergraduates' brand new beauty algorithm boarded on ACM SIGGRAPH

Figure 2: The flow chart of the research, please refer to the original paper for details. 

Result display

The study tested the performance of the method on a large number of portrait images with different genders, postures, face shapes, and skin tones. Figure 3 shows the results automatically generated by the method proposed in this research. 

Taking double chin removal as an example, this method can successfully remove the double chin of the input portrait image while keeping other features unchanged. 

There is a strange trick to remove double chin, Zhejiang University post-00s undergraduates' brand new beauty algorithm boarded on ACM SIGGRAPH

Figure 3: Results of this study. The first four rows are the results of continuous adjustment of parameters. In each pair of images in the last four rows, the left picture is the original picture, and the right picture is the result obtained. 

Compared with the current optimal face editing method (SOTA), this research produces more stable and reasonable results, maintains the invariance of facial features, and conforms to the face structure. 

There is a strange trick to remove double chin, Zhejiang University post-00s undergraduates' brand new beauty algorithm boarded on ACM SIGGRAPH

Figure 4: Comparison of methods. The first line is the input portrait image, the second line is the result of MaskGAN, the third line is the result of SC-FEGAN, the fourth line is the result of the Generative Inpainting method, and the last line is the result of our method. 

The researchers hope that this research will bring new ideas to face editing, and at the same time hope to bring inspiration to StyleGAN’s hidden space research. 

Posted by:CoinYuppie,Reprinted with attribution to:https://coinyuppie.com/there-is-a-strange-trick-to-remove-double-chin-zhejiang-university-post-00s-undergraduates-brand-new-beauty-algorithm-boarded-on-acm-siggraph/
Coinyuppie is an open information publishing platform, all information provided is not related to the views and positions of coinyuppie, and does not constitute any investment and financial advice. Users are expected to carefully screen and prevent risks.

Like (0)
Donate Buy me a coffee Buy me a coffee
Previous 2021-07-27 09:09
Next 2021-07-27 09:11

Related articles