Understanding and coding Neural Networks in Python Using Keras

When we discuss "neural network", we mean the artificial neural network (ANN). The idea of ANN is based on the biological nervous network such as the brain.

First, we start the discussion with how a biological neuron works?

A biological neuron contains 3 mai parts 1) cell  body 2)Dendrites 3) Axons

credits: https://simple.wikipedia.org/wiki/Neuron

The biological neuron can be represented as the figure below, the input(x1, x2, x3, ..., xn) is the dendrites and axons are the output from a neuron and in the middle, it is cell body which can process the input.

credits: https://www.python-course.eu/neural_networks.php

It is surprisingly simple, what is going on inside the neuron's body. Input signals are multiplied by weight values, i.e. weighing in each input. In this way, input can be individually adjusted for each Xi. We can see the same weight as all input and weight vectors in the form of an input vector W.

When a signal arrives, it gets multiplied by the weight value assigned to that particular input. That is, if a neuron has three inputs, then there are three weights that can be adjusted individually. Weight usually gets adjusted during the learning phase.Weights will be multiplied with inputs in the manner (w1*x1 + w2*x2 + w3*x3 + 1*b). This sum will be given to an activation function which is generally a sigmoid function. 

In the brain, these neurons will form a network which we called as a neural network (by connecting with other neurons) and we will form an artificial neural network (by connecting artificial neurons) with artificial neurons. This is also called as multilayer perceptron.

Credits: http://neuralnetworksanddeeplearning.com/chap1.html

Forward Propagation and Backward Propagation:

Forward propagation is finding the output of the network by assuming weight vector W randomly. We will calculate the output as f(W*X). If we are far away from the original output we will use Backpropagation to modify the weight vector W to gain the output near to the original output.

Backpropagation algorithm works by finding loss function in the forward propagation and propagating it back into the network. The weights are updated to minimize the error in the network. Backpropagation will discuss in full detail in future.

Implementing a neural network using python with keras library:

#Importing Libraries
from keras.utils import np_utils 
from keras.datasets import mnist 
import seaborn as sns
from keras.initializers import RandomNormal
from keras.models import Sequential 
from keras.layers import Dense, Activation

#Loading data and converting into a one dimensional data 
(X_train, y_train), (X_test, y_test) = mnist.load_data()
# if you observe the input shape its 3 dimensional vector
# for each image we have a (28*28) vector
# we will convert the (28*28) vector into single dimensional vector of 1 * 784 
X_train = X_train.reshape(X_train.shape[0], X_train.shape[1]*X_train.shape[2]) 
X_test = X_test.reshape(X_test.shape[0], X_test.shape[1]*X_test.shape[2]) 

# Data normalization
X_train = X_train/255
X_test = X_test/255

#Converting output labels into categorical features and output label 5 represent as [ 0.  0.  0.  0.  0.  1.  0.  0.  0.  0.]
Y_train = np_utils.to_categorical(y_train, 10) 
Y_test = np_utils.to_categorical(y_test, 10)

# some model parameters
output_dim = 10
input_dim = X_train.shape[1]
batch_size = 128 
nb_epoch = 20

#single perceptron
model = Sequential()
model.add(Dense(output_dim, input_dim=input_dim, activation='softmax'))
#compiling the model 
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# Training the model with train data and evaluating using test data
model.fit(X_train, Y_train, batch_size=batch_size, epochs=nb_epoch, verbose=1, validation_data=(X_test, Y_test))

       
 

That's it, you are all done with the code of neural network(single perceptron). This code will run up to 20 epochs and the accuracy we get is > 90%. Looking so awesome right!!!. greater than 90% of accuracy with a just single neuron. That is the power of artificial neural networks. We can achieve the accuracy up to 90% also by selecting a correct network.

After 20 epochs the output will be looking like this with train accuracy=93.18% and Test accuracy = 92.;7

Epoch 20/20
60000/60000 [==============================] - 1s 19us/step - loss: 0.2456 - acc: 0.9318 - val_loss: 0.2633 - val_acc: 0.9277


If we plot the graph between Train, Test Accuracy and no of epochs ... it will be like this.

I hope you now understood the basic terminology involved in neural networks and the implementation of the neural network using the most popular keras library. In future articles we will discuss in detail about the deep learning concepts. So stay with my blog to learn all the machine learning and deep learning implementations and coding part  using the most popular libraries tensorflow and keras.

My next articles will be on 1) implementing multilayer neural network 

2) convolutional Neural Networks

3) Gradient Descent algorithm and how backpropagation works