- Discriminative models learn to find the decision boundary that separates the classes in an optimal way, while generative models learn about the characteristics of each class. That is, discriminative models predict the labels conditioned on the input,
, whereas generative models learn the joint probability distribution, 
. - The generator learns the distribution of images in our dataset. It learns the distribution of handwritten digits in our training set. We feed random noise to the generator and it will convert the random noise into a new handwritten digit similar to the one in our training set.
- The goal of the discriminator is to perform a classification task. Given an image, it classifies it as real or fake; that is, whether the image is from the training set or the one generated by the generator.
-
The loss function for the discriminator is given as follows:
The generator loss function is given as follows:
- DCGAN extends the design of GANs with convolutional networks. That is, we replace with feedforward network in the generator and discriminator with the Convolutional Neural Network (CNN).
-
The Kullback-Leibler (KL) divergence is one of the most popularly used measures for determining how one probability distribution diverges from the other. Let's say we have two discrete probability distributions,
and 
, then the KL divergence can be expressed as follows:
- The Wasserstein distance, also known as the Earth Movers (EM) distance, is one of the most popularly used distance measures in the optimal transport problems where we need to move things from one configuration to another.
- A Lipschitz continuous function is a function that must be continuous and almost differentiable everywhere. So, for any function to be a Lipschitz continuous, the absolute value of a slope of the function’s graph cannot be more than a constant,
. This constant, 
, is called the Lipschitz constant.