Discriminator and generator loss

Given the setup that we have described,  and  play a iterative two-player minimax game with the value function :

Literally speaking, it minimizes its loss when D(x) is equal to 1 and D(G(z)) is equal to 0, that is, when the discriminator's probability of real is 1 for real data and 0 for fake data. Hence, the discriminator is maximizing the outputs from D(X) and D(G(z)).

The generator, on the other hand, minimizes its loss when D(G(z)) is equal to 1, that is, when the discriminator's probability of real data as  fake data is 1. Hence, the generator is minimizing the outputs from D(G(z)).

The objective function described in the preceding equation is equivalent to minimizing the Jensen-Shannon (JS) divergence between the distributions, as described in the paper Generative Adversarial Nets by Ian Goodfellow et al. The JS divergence is a symmetric and smoothed version of the Kullback-Leibler (KL) divergence, described in the paper On information and sufficiency by Kullback and Leibler. Note that in the following equation,  stands for divergence, not discriminator:

 on a continuous support is defined as follows:

A closer look at the KL divergence described earlier shows a few problems with respect to exploding losses when the support of P is not contained in Q. In the next section, we will address this topic within the context of the strengths and weaknesses of GANs.