Understanding Generative Adversarial Networks (GANs): An Easy Guide in 5 min read

Generative Adversarial Networks, or GANs, have become a revolutionary tool in the field of artificial intelligence (AI). Introduced by Ian Goodfellow and his colleagues in 2014, GANs have since enabled machines to create remarkably realistic images, videos, and even sounds. In this article, we’ll break down the concept of GANs in simple terms and provide examples to illustrate how they work.

What are GANs?

GANs are a type of neural network architecture used for generative tasks. In simpler terms, they can generate new data that is similar to existing data. A GAN consists of two main components:

1. Generator: This network generates new data.
2. Discriminator: This network evaluates the generated data to determine if it’s real (from the original dataset) or fake (created by the generator).

These two networks are like two players in a game. The generator tries to fool the discriminator by creating realistic data, while the discriminator tries to get better at distinguishing real data from fake data.

The Generator

The generator’s primary goal is to produce data that is as realistic as possible. Here’s a detailed breakdown of its workings:

1. Input Noise: The generator takes a random noise vector (often sampled from a Gaussian or uniform distribution) as input. This noise vector is essentially a set of random numbers.
2. Transformation: The generator transforms this noise into a new piece of data through a series of layers, which can include dense layers, convolutional layers, and upsampling layers. During this process, the generator learns to map the random noise to realistic data samples.
3. Output Data: The output of the generator is a data sample that ideally resembles the real data. For example, if the GAN is trained on images of handwritten digits, the generator will output an image of a digit.
4. Learning Process: The generator’s parameters are updated based on feedback from the discriminator. When the discriminator correctly identifies the generator’s output as fake, the generator receives a signal to adjust its weights to produce more realistic data in future iterations.

The Discriminator

The discriminator’s job is to distinguish between real and fake data. It acts as a binary classifier, outputting a probability that indicates whether a given data sample is real or fake. Here’s a detailed breakdown of its workings:

1. Input Data: The discriminator takes an input image, which can be either a real image from the training dataset or a fake image generated by the generator.
2. Feature Extraction: Through a series of convolutional layers, the discriminator extracts features from the input image. These layers help the discriminator understand the underlying patterns and structures of the images.
3. Classification: The final layer of the discriminator is typically a dense layer with a sigmoid activation function, which outputs a probability score between 0 and 1. A score close to 1 indicates that the input image is real, while a score close to 0 indicates that it is fake.
4. Learning Process: The discriminator’s parameters are updated based on its classification performance. It receives feedback when it misclassifies an image, and this feedback is used to improve its ability to distinguish between real and fake images.

How Do GANs Work?

Imagine GANs as a forger and an art critic. The forger, which we call the generator, creates fake paintings, while the art critic, known as the discriminator, tries to tell which paintings are real and which are fake. Over time, both the forger and the critic improve their skills until the fake paintings are almost indistinguishable from the real ones.

Here’s a step-by-step explanation:

Initial Phase

1. Generator Starts with Noise:
   • The generator begins by taking a random noise vector as input. This noise is typically a random array of numbers sampled from a Gaussian or uniform distribution.
   • Using this noise, the generator produces an image. Initially, this image is just a random pattern, but it will improve over time.
2. Discriminator Gets Real and Fake Images:
   • The discriminator is presented with two types of images: real images from the dataset and fake images produced by the generator.
   • The real images come from a training set, such as the MNIST dataset, which contains images of handwritten digits.

Discriminator Training

1. Discriminator Distinguishes Real from Fake:
   • The discriminator is a neural network that takes an image as input and outputs a probability score indicating whether the image is real (close to 1) or fake (close to 0).
   • During training, the discriminator is given real images labeled as 1 and fake images labeled as 0.
2. Feedback Loop for Discriminator:
   • The discriminator adjusts its weights through backpropagation to minimize its loss, which is a measure of how well it distinguishes real images from fake ones.
   • The loss function typically used is binary cross-entropy, which provides a measure of the difference between the predicted probability and the actual label.

Generator Training

1. Generator Aims to Fool the Discriminator:
   • The generator’s goal is to create images that are realistic enough to fool the discriminator into thinking they are real.
   • The generator is trained using the feedback from the discriminator’s ability to distinguish real from fake images.
2. Feedback Loop for Generator:
   • The generator adjusts its weights to minimize its loss, which is derived from how well it can deceive the discriminator.
   • Instead of getting direct feedback on the quality of the images, the generator gets feedback based on the discriminator’s output. This is done by defining a generator loss that uses the discriminator’s output.

Iterative Process

1. Alternating Training:
   • The training process involves alternating between training the discriminator and the generator. This can be done in the following steps:
     • Train the discriminator on a batch of real images and a batch of fake images.
     • Train the generator on a batch of noise vectors, using the feedback from the discriminator to improve its output.
2. Mutual Improvement:
   • With each iteration, the generator becomes better at creating realistic images, as it learns from the discriminator’s feedback.
   • Simultaneously, the discriminator improves its ability to distinguish real images from fakes, as it learns from both real images and the increasingly realistic fake images generated by the generator.
3. Convergence:
   • Ideally, this process continues until the generator produces images so realistic that the discriminator cannot reliably tell them apart from real images.
   • At this point, the discriminator’s accuracy hovers around 50%, indicating that it is as likely to classify a generated image as real as it is to classify a real image as fake.

Visualization of the GAN Training Process

Here’s a simple visualization of the GAN process:

Example: Generating Handwritten Digits

In this detailed example, we will walk through the process of building and training a Generative Adversarial Network (GAN) to generate handwritten digits similar to those in the MNIST dataset. We’ll provide Python code using TensorFlow and Keras, making it easier for engineers to follow along and implement their own GAN.

Step 1: Data Preparation

First, we need to load the MNIST dataset, which contains images of handwritten digits from 0 to 9.

import tensorflow as tf
from tensorflow.keras.datasets import mnist
import numpy as np

# Load the MNIST dataset
(x_train, _), (_, _) = mnist.load_data()

# Normalize the images to the range [-1, 1]
x_train = (x_train.astype(np.float32) - 127.5) / 127.5
x_train = np.expand_dims(x_train, axis=-1)

# Print the shape of the dataset
print(x_train.shape)  # Output: (60000, 28, 28, 1)

Step 2: Building the Discriminator

The discriminator is a neural network that takes an image as input and outputs a single value representing the probability that the image is real.

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, LeakyReLU, Conv2D, Dropout

def build_discriminator():
    model = Sequential()

    model.add(Conv2D(64, kernel_size=3, strides=2, input_shape=(28, 28, 1), padding="same"))
    model.add(LeakyReLU(alpha=0.2))
    model.add(Dropout(0.3))

    model.add(Conv2D(128, kernel_size=3, strides=2, padding="same"))
    model.add(LeakyReLU(alpha=0.2))
    model.add(Dropout(0.3))

    model.add(Flatten())
    model.add(Dense(1, activation='sigmoid'))

    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

    return model

discriminator = build_discriminator()
discriminator.summary()

Step 3: Building the Generator

The generator is another neural network that takes random noise as input and generates an image as output.

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Reshape, BatchNormalization, UpSampling2D, Conv2D, LeakyReLU

def build_generator():
    model = Sequential()

    model.add(Dense(128 * 7 * 7, activation="relu", input_dim=100))
    model.add(Reshape((7, 7, 128)))
    model.add(BatchNormalization())



    model.add(UpSampling2D())
    model.add(Conv2D(128, kernel_size=3, padding="same"))
    model.add(LeakyReLU(alpha=0.2))

    model.add(UpSampling2D())
    model.add(Conv2D(64, kernel_size=3, padding="same"))
    model.add(LeakyReLU(alpha=0.2))

    model.add(Conv2D(1, kernel_size=3, padding="same", activation='tanh'))

    return model

generator = build_generator()
generator.summary()

Step 4: Building and Compiling the GAN

We now combine the generator and discriminator into a GAN. The discriminator is set to be non-trainable when combined with the generator to ensure it remains static when the generator is training.

from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input

# Build and compile the discriminator
discriminator = build_discriminator()

# Build the generator
generator = build_generator()

# Create the GAN by stacking the generator and discriminator
z = Input(shape=(100,))
img = generator(z)
discriminator.trainable = False
validity = discriminator(img)

gan = Model(z, validity)
gan.compile(loss='binary_crossentropy', optimizer='adam')

Step 5: Training the GAN

We train the GAN by alternating between training the discriminator and the generator. In each iteration, we generate a batch of fake images, use them to train the discriminator along with real images, and then train the generator to fool the discriminator.

import matplotlib.pyplot as plt

def train(epochs, batch_size=128, save_interval=1000):
    half_batch = batch_size // 2

    for epoch in range(epochs):

        # Train Discriminator
        idx = np.random.randint(0, x_train.shape[0], half_batch)
        real_imgs = x_train[idx]

        noise = np.random.normal(0, 1, (half_batch, 100))
        fake_imgs = generator.predict(noise)

        d_loss_real = discriminator.train_on_batch(real_imgs, np.ones((half_batch, 1)))
        d_loss_fake = discriminator.train_on_batch(fake_imgs, np.zeros((half_batch, 1)))
        d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)

        # Train Generator
        noise = np.random.normal(0, 1, (batch_size, 100))
        valid_y = np.array([1] * batch_size)

        g_loss = gan.train_on_batch(noise, valid_y)

        # Print the progress
        print(f"{epoch} [D loss: {d_loss[0]} | D accuracy: {100 * d_loss[1]}] [G loss: {g_loss}]")

        # Save generated images every save_interval
        if epoch % save_interval == 0:
            save_imgs(epoch)

def save_imgs(epoch):
    r, c = 5, 5
    noise = np.random.normal(0, 1, (r * c, 100))
    gen_imgs = generator.predict(noise)

    gen_imgs = 0.5 * gen_imgs + 0.5  # Rescale images 0 - 1

    fig, axs = plt.subplots(r, c)
    cnt = 0
    for i in range(r):
        for j in range(c):
            axs[i, j].imshow(gen_imgs[cnt, :, :, 0], cmap='gray')
            axs[i, j].axis('off')
            cnt += 1
    fig.savefig(f"gan_images/mnist_{epoch}.png")
    plt.close()

# Train the GAN for 10,000 epochs
train(epochs=10000, batch_size=64, save_interval=1000)

This code will train the GAN to generate images of handwritten digits. The save_imgs function saves the generated images at regular intervals so you can see the progress. By following these steps and using the provided code, you can build and train a GAN to generate images similar to those in the MNIST dataset. The process involves preparing the data, building the discriminator and generator networks, combining them into a GAN, and training the GAN by alternating between training the discriminator and the generator. This hands-on approach helps you understand how GANs work and how they can be used to generate new data.

Real-World Applications of GANs

GANs have a wide range of applications beyond generating handwritten digits. Here are a few notable examples:

Image Generation

GANs can generate high-resolution images from text descriptions. For example, given a description of a bird, a GAN can create a realistic image of that bird. This application has immense potential in creative industries and entertainment, enabling the creation of realistic visuals from simple text prompts.

Image Enhancement

GANs can be used to enhance image quality, such as increasing the resolution of low-quality images or restoring old photographs. This technology, often referred to as super-resolution, has significant implications for photography, forensic analysis, and satellite imaging.

Art and Design

Artists and designers use GANs to create new artwork, including paintings, music, and fashion designs. GANs can generate novel patterns and designs that inspire new creative works. For instance, GANs have been used to generate unique clothing designs and even assist in composing music.

Medical Imaging

In the medical field, GANs are used to create synthetic medical images for research and training purposes, helping doctors and researchers develop new diagnostic tools and treatments. GANs can generate images of various medical conditions, aiding in the development of better diagnostic algorithms and training datasets.

Video Game Development

GANs are employed to create realistic textures and environments in video games, enhancing the visual experience for players. By generating high-quality assets and environments, GANs can significantly reduce the time and cost associated with game development.

Conclusion

Generative Adversarial Networks are a powerful and versatile tool in AI, capable of creating highly realistic data. By pitting a generator against a discriminator, GANs learn to produce new data that mimics real-world examples. Whether it’s generating images, enhancing photos, or creating art, GANs are opening up new possibilities across various fields.

Understanding GANs doesn’t require advanced technical knowledge—just a basic grasp of how two neural networks can improve through competition. As GAN technology continues to evolve, we can expect even more impressive and innovative applications in the future. GANs are not only transforming the way we generate data but also pushing the boundaries of what is possible with artificial intelligence.