Generative Adversarial Networks(GANs) part 1

Generative adversarial networks (GANs) are the most exciting recent innovation in the machine learning area. Generative Adversarial Networks, A deep learning method such as convolutional neural networks, an approach to generative modeling: creates new sample data instances that resemble training data. GANs can create images that look like photographs of human faces, even though the faces don't belong to any real person.

In this tutorial, we will discuss very basic learning materials for GANs.

Table of contents

Machine Learning 

Supervised Learning 

Unsupervised Learning 

Reinforcement Learning

Deep Neural Network 

Generative modeling

Gaussian Distribution

Dimensionality Reduction

Encoder 

Decoder 

Latent Space 

Autoencoder 

Autoencoder and Prism

Variational Autoencoder

Machine Learning 

Machine Learning is the study of computer algorithms or a method of data analysis that makes the computer ability to learn. It is a branch of Artificial Intelligence. It automates the analytical model building. Machine Learning is one of the most exciting technologies that would have ever come across. It focuses on the development of computer algorithms that can 

• Learn from data 

• Identify patterns 

• Make predictions with minimal human intervention 

• Improve automatically through experience 

We can categorize the most widely adopted machine learning methods as supervised learning, unsupervised learning, and reinforcement learning. We will briefly discuss these in the following tutorial.

Supervised Learning 

Supervised Learning is the machine learning task where machines are learned from “labeled data” and predict outcomes for unseen data.  It learns a mapping function that maps an input to an output based on training examples. In training, we give both input and output to the function which maps an input to output and adopts to learn for unseen input.  

The learning task is called supervised learning because learning from the labeled dataset can be thought of as a teacher supervising the learning process. We know the correct answers, the algorithm iteratively makes predictions on the training data and is corrected by the teacher. Look at the example given below:

Figure: Working process of supervised learning Image source

Suppose we have a dataset of different types of shapes which includes square, rectangle, triangle, and Polygon. Now the first step is that we need to train the model for each shape.

• If the given shape has four sides, and all the sides are equal, then it will be labeled as a Square.
• If the given shape has three sides, then it will be labeled as a triangle.
• If the given shape has six equal sides, then it will be labeled as a hexagon.

Now, after training, we test our model using the test set, and the task of the model is to identify the shape.

The machine is already trained on all types of shapes, and when it finds a new shape, it classifies the shape based on the number of sides and predicts the output.

Unsupervised Learning 

Unsupervised Learning is the machine learning task where machines are learned from “unlabeled data” and learns some pattern from data. In unsupervised learning, we have only input data and the goal is to find the underlying structures or patterns or groups of input data. It can discover similarities and differences in data for making a realistic assumption about data. In unsupervised learning, training examples are not labeled, and finds the patterns without any supervision. Clustering, for example, is an unsupervised learning example—because we are just trying to discover the underlying structure of the data; but anomaly detection is usually supervised, as we need human-labeled anomalies. Look at the example given below:

Figure: Working process of unsupervised learning Image Source

Here, we have taken unlabeled input data, which means it is not categorized and corresponding outputs are also not given. Now, this unlabeled input data is fed to the machine learning model to train it. Firstly, it will interpret the raw data to find the hidden patterns from the data and then will apply suitable algorithms such as k-means clustering, Decision tree, etc.

Once it applies the suitable algorithm, the algorithm divides the data objects into groups according to the similarities and differences between the objects.

Reinforcement Learning

Deep Neural Network 

The neural network is a series of algorithms that are designed to recognize patterns or underlying relationships in the input data. A neural network is a network made up of biological neurons for solving artificial intelligence (AI) problems. It is a collection of interconnected units or nodes called neurons. Every connection like the synapses in a biological brain can transmit the signal to another connected neuron and these neurons received the transmitted signal and processed it.  

The network consists of various layers where each layer has multiple neurons. The neurons or nodes are interconnected to one another in various layers. The first layer is called the input layer and the last layer is called the output layer and the intermediate layer is called the hidden layer. Each neuron performs complex computation and passes the result to connected neurons. Then the output layer transforms the result into the desired output. 

 Figure: Basic diagram of neural networks

Generative Modeling

We are familiar with how deep learning takes raw pixels and turns them into, for example, class predictions. For example, we can take three matrixes that contain pixels of an image (one for each color channel) and pass them through a system of transformations to get a single number at the end. But what if we want to go in the opposite direction? What if we want to generate new images instead of predictions? Here comes the idea of generative modeling. A generative model describes how a dataset is generated, in terms of a probabilistic model. By sampling from this model, we are able to generate new sample data. More formally, we take a certain image (real sample) and try to generate new samples from it which look as realistic as the real sample.

Figure: Process of generative modeling Image source

Encoder 

An encoder network is a one or many-layer neural network that compresses high-dimensional input data into a lower-dimensional representation vector. We take an input representation x and then a learned encoder reduces the dimension from x1 to x2 where x1 > x2 and generates an output representation z. An encoder network is given below:

Figure: Encoder Network

Decoder 

A decoder network is opposite to an encoder network. It is also a one or many-layer neural network that decompresses a given representation vector back to the original domain. We take a representation z and then a learned decoder enhances the dimension from x2 to x1 where x2 < x1 and produces an output representation x*. A decoder network is given below:

Figure: Decoder Network

Latent Space

Latent space is the hidden representation of compressed data. It is a learned representation, meaningful to people in ways we think of it. In other words, Latent space refers to an abstract multi-dimensional vector space containing feature values that we cannot interpret directly, but which encodes a meaningful internal representation of externally observed events.

Different models will learn a different latent representation of the same data. Typically, it is a 100-dimensional hypersphere with each variable drawn from a Gaussian distribution with a mean of zero and a standard deviation of one.

Let's dive into autoencoder, a generative model.

Autoencoder

Autoencoder and Prism

In optics, some piece of glass cut with a specific shape, useful for analyzing and reflecting light, is called prism. An ordinary prism can separate white light into its constituent colors, called a spectrum. Generally, seven colors or spectrums we can extract from a white light source passing through a prism.

Figure: The white light passing through a prism and divided into 7 spectrums

On the other hand, we can merge these 7 spectrums to get white color. Seven spectrums pass through the prism and the prism combines these spectrums and constructs a white light source.

Figure: The seven spectrums passing through a prism and these spectrums are converted into white light

We can also combine these two processes. For this, we place two prisms. After that, we pass seven spectrums through the first prism. The first prism combines these seven spectrums into one white light source. Then the white light source passes through the second prism. The second prism extracts seven spectrums back again.