GANs
Generative Adversarial Networks - a two NN competition
Last updated
Generative Adversarial Networks - a two NN competition
Last updated
The key contribution of GANs is to model the learning process as a zero-sum game between two neural networks
The first network is called the generator and is intended to generate samples that match the distribution of the training dataset
The second neural network is known as the discriminator and examines samples to determine whether they are real or fake
The discriminator learns using traditional supervised learning techniques, dividing inputs into two classes: real or fake
In the GAN model, the generator is trained to fool the discriminator and tries to determine whether the data produced by the generator is real or not
The competition created by the GAN model should drive both the generator and discriminator to improve their learning policies
A generator takes as input a random distribution (also known as ‘noise’ in GANs literature), and learns a mapping function that maps the input to the desired output, which is the actual underlying distribution of the data.
We will use a second NN known as discriminator.
The discriminator’s role is to judge and assess the quality of output images from the generator
Technically, the discriminator is a binary classifier.
It accepts images as input, and outputs a probability that the image is real (i.e. actual training image), or fake (i.e. from the generator).
Initially, the generator struggles to produce images that look real, and the discriminator can easily distinguish real and fake images without making too many mistakes
Since the discriminator is a binary classifier, we can quantify the performance of the discriminator using the Binary Cross-Entropy (BCE) Loss
The discriminator’s BCE loss is an important signal for the generator. Recall earlier, that by itself the generator doesn’t know if the generated images resemble the real images. However, the generator can use the discriminator’s BCE loss as a signal to obtain feedback for its generated images.
Here’s how it works.
We send images output by the generator to the discriminator and it predicts the probability that the image is real.
Initially, when the generator is poor, the discriminator can easily classify the images as fake, resulting in a low BCE loss. This info is sent to the generator to improve its neural network weights.
The generator eventually improves and the discriminator starts to make more mistakes, misclassifying the fake images as real, which results in a higher BCE loss.
Since the discriminator is a binary classifier, its training procedure is straightforward
We’ll provide a batch of labelled real and fake images to the discriminator, and we’ll use the BCE loss to tune the weights of the discriminator
We train the discriminator to identify real vs fake images, preventing the discriminator from being ‘fooled’ by the generator
In practice, we train the generator and discriminator in turns, one after another
This training scheme is akin to a two-player minimax adversarial game, as the generator aims to maximize the discriminator’s loss, and the discriminator aims to minimize its own loss
This process stops when it coverges (we can say the the discriminator reaches a 50% chances to discriminate real or fake images)
