High Resolution Face2Face with Pix2Pix 1024x1024

Inspired by this work Dat Tran, I prepared my own dataset and trained improved Pix2Pix net to generate Polish youtuber Krzysztof Gonciarz creating show “Zapytaj Beczkę”.

But first let’s give it a try:

To increase resolution one needs to add layers to encoder and decoder, there is no simpler way to do it. So I added few layers and tuned parameters to fit my 8GB memory on GTX980M in my laptop.

If you need to modify input/output size just add or remove some layers in encoder and decoder:

Encoder before (ngf = 64)

Encoder after (ngf = 64)

I used 4 videos as input for training:

I used frames with one face detected from every 10th frame of these videos and manually removed some bad examples (e.g. other people).

Finally after 3 days of training (GTX980M) I get these results:

You can find source code on my GitHub:

The original model (with Angela Merkel) was working good only in one position and distance from camera. Here I have similar problem — face was very close to the camera in all training videos. If you are going to train net to generate faces — remember to prepare very good dataset. The increase of input/output resolution was really good idea!

From my experience it was super easy to train this net and get first outputs. Totally different than with YOLO/SSD networks, where you need images and annotations in specified format. I will write when I successfully train YOLO or SSD!

Any ideas what to do next? Stay tuned for more deep learning posts!

Please check Dat Tran stories:

If you like this work show me your support!
Follow me on:

Related posts:

The Sorcery gem is a self-described “stripped-down, bare-bones authentication library” which I recently familiarized myself with while adding Authorization/Authentication to a Rails app that I built…

I just started learning Docker today and I was ducking confused by both of those definition. Soooooo this is my attempt to summarize my learning for myself and for others. If I’m wrong about anything…