Neural Style Transfer: Transforming My Face Into a Painting

The Neural Style Transfer (NST) algorithm was created by Gatys et al. (2015). NST is probably one of the most fun techniques in deep learning. It combines two images namely: content image and a style image to create a generated image. The generated image is a combination of the content of the image with that of the the style of the image.

Here's everything there is to remember when implementing NST using tensorflow.

1. Load the content image. I chose a random image of myself. 

I created an interactive session in TensorFlow. Unlike a regular session, the "Interactive Session" installs itself as the default session to build a graph. This allows us to run the variables without constantly needing to refer to the session object (calling "sess.run()"), which simplifies the code.

2. Load the style image. I chose the famous scream painting by Edvard Munch. I cropped it so that the dimensions match my image above.

3. Randomly initialize the image to be generated. This will look more or less like the image below.

4. Load the VGG19 model. 

Neural Style Transfer (NST) uses a previously trained convolutional network, and builds on top of that. The idea of using a network trained on a different task and applying it to a new task is called transfer learning. This is great because I don't need to train a model for several hours or days on a laptop!

Following the original NST paper, for this task, I used the VGG network. Specifically, VGG-19, a 19-layer version of the VGG network. This model has already been trained on the very large ImageNet database, and thus has learned to recognize a variety of low level features (at the shallower layers) and high level features (at the deeper layers).

5. Build the TensorFlow graph. Initialize the TensorFlow graph and run it for a large number of iterations, updating the generated image at every step.

I captured the resulting image every 10 iteration steps. The result is shown below.

This exercise was part of the Deep Learning Specialization by deeplearning.ai.