Hello everyone, welcome to my Blog in this tutorial, we will be learning how to do simple image classification as we did in the last tutorial but this time, we will be leveraging PyTorch Lightning to do the same task but with ease. So, I am planning to release both core PyTorch implementation as well as PyTorch Lightning implementation.

Without much due, let's dive directly into the code.

In Local Machine

    pip3 install lightning

In Colab Notebook

    !pip install lightning

At first, we need to install lightning in our device. For this tutorial, we will be using Colab, and in colab also we have to install it.

import os
import torch
from torch import nn
import torchvision
import torch.nn.functional as F
from torchvision import transforms
from torch.utils.data import DataLoader
import lightning.pytorch as pl

So, all the required modules are imported. Everything looks almost the same, just the import of lightning.pytorch as pl is new. Then, let's copy a few more lines from previous tutorials, nothing new, the same lines and the same function.

# Data preprocessing and augmentation
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
])

# Load CIFAR-10 dataset
dataset = torchvision.datasets.CIFAR10(root='./cifar', train=True, transform=transform, download=True)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=64, shuffle=True)

criterion = nn.CrossEntropyLoss()

The model definition will also be the same.

class SimpleANN(nn.Module):
    def __init__(self):
        super(SimpleANN, self).__init__()
        self.fc1 = nn.Linear(3 * 32 * 32, 128)  # Flatten the 32x32 RGB images
        self.fc2 = nn.Linear(128, 64)
        self.fc3 = nn.Linear(64, 10)  # Output layer for 10 classes

    def forward(self, x):
        x = x.view(-1, 3 * 32 * 32)  # Flatten the input images
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

class LitSimpleANN(pl.LightningModule):
  def __init__(self, model):
    super().__init__()
    self.model = model

  def training_step(self, batch, batch_idx):
    x, y = batch
    # print(y)
    out = self.model(x)
    # print(out)
    loss = criterion(out, y)
    preds = torch.argmax(out, dim=1)
    accuracy = torch.sum(preds == y).item() / y.size(0)

    self.log('train_loss', loss,  on_epoch=True)  # Log training loss
    self.log('train_accuracy', accuracy,  on_epoch=True)
    return loss

  def configure_optimizers(self):
    optimizer = torch.optim.SGD(self.model.parameters(), lr=0.001, momentum=0.9)
    return optimizer

Now, the real magical lines of PyTorch lightning come into action. We defined a class LitSimpleANN as a subclass of PyTorch Lightning Module and this class takes the above-defined SimpleANN class as input. Then, we define the training step inside this LitSimpleANN class. training_step function takes batch and batch_idx as input. So each batch has input images and the labels associated with the batch. So, Same as before, we fed the model the input image, it gives the probability distribution of each class and we calculate loss and accuracy on it. Here, self.log stores loss and accuracy per step and per epoch which can be useful during the evaluation of the model performance.

model = LitSimpleANN(SimpleANN())

Training the data

from pytorch_lightning.loggers.csv_logs import CSVLogger

trainer = pl.Trainer(max_epochs=10,  logger=CSVLogger("logs"))
trainer.fit(model = model, train_dataloaders=dataloader)

Here, I have imported CSVLogger to save the logs in CSV format for further reference. Here, pl.Trainer takes away all the manual training loops from the custom PyTorch training loop we also discussed in the last video. So, when we hit shift enter, Bam, the training starts. Nothing much fancy here, max_epochs=10 is for the number of epochs, as the default is 1000 epochs. The output of this code is as follows:

Let's plot some graphs.

import pandas as pd
import seaborn as sn
metrics = pd.read_csv(f"{trainer.logger.log_dir}/metrics.csv")
del metrics["step"]
metrics.set_index("epoch", inplace=True)
sn.relplot(data=metrics, kind="line")

The mertics.csv is auto-generated by trainer logger and this will give a nice line graph.

Conclusion

In this post, we learned how to implement image classification using simple ANN in the PyTorch Lightning library. Lightening AI is fast, easy to use and saves us from unnecessary loops of code. I highly recommend you all learn about it and it will save you from hassle compared to core PyTorch code.

This much from today, so see you guys in the next tutorial.

Colab

YouTube

Bibek Chalise is a Machine Learning enthusiast, Computer Vision Scientist and is associated with MarginTop Solutions.

MarginTop Solutions

Where Tech Meets Brilliance

Pokhara, Nepal

margintopsolutions@gmail.com

Website

Hello everyone, welcome to my Blog in today's tutorial, we will be designing a more advanced ANN for image classification. So, this will be the subsequent part of our series on Computer Vision With PyTorch. So without a due, let's dive into the code.

So, this part of our code will be the same till download our dataset and create a data loader. If you are unclear about any step, I highly recommend you to check the tutorial, where I have explained all about it.

import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms
from torch.utils.data import DataLoader
import matplotlib.pyplot as plt

# Data preprocessing and augmentation
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
])

# Load CIFAR-10 dataset
train_dataset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

Now, we need to define our model which will be structured in the same way as was previous tutorial but will have more layers.

class AdvancedANN(nn.Module):
    def __init__(self):
        super(AdvancedANN, self).__init__()
        self.fc1 = nn.Linear(32 * 32 * 3, 1024)
        self.fc2 = nn.Linear(1024, 512)
        self.fc3 = nn.Linear(512, 256)
        self.fc4 = nn.Linear(256, 128)
        self.fc5 = nn.Linear(128, 64)
        self.fc6 = nn.Linear(64, 10)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = x.view(x.size(0), -1)  # Flatten the input images
        x = self.relu(self.fc1(x))
        x = self.relu(self.fc2(x))
        x = self.relu(self.fc3(x))
        x = self.relu(self.fc4(x))
        x = self.relu(self.fc5(x))
        x = self.fc6(x)
        return x

Here, our 3*32*32 image is converted to a 1-dimensional vector of 3072*1 and feed into the network. Then, it is projected to 1024 nodes followed by 512, 256, 128, 64 nodes and finally we have projected to 10 nodes because we have 10 classes in our dataset. The same is in the forward function, the view function is for reshaping the image into a single dimension, that's how we do reshaping in PyTorch.

# Hyperparameters
batch_size = 64
learning_rate = 0.001
num_epochs = 10

# Initialize the model
model = AdvancedANN()
# Define loss function and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=learning_rate)

This is also the same as the previous, just we will be using Adam as optimizer instead of SGD. Now, we need to define our training loop.

losses = []
accuracies = []
# Training loop
for epoch in range(num_epochs):
    model.train()  # Set the model to training mode
    total_correct = 0
    total_samples = 0
    total_loss = 0.0
    for i, (images, labels) in enumerate(train_loader):
        optimizer.zero_grad()
        outputs = model(images)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        _, predicted = torch.max(outputs, 1)
        total_samples += labels.size(0)
        total_correct += (predicted == labels).sum().item()
        total_loss += loss.item()

        if (i + 1) % 100 == 0:
            print(f"Epoch [{epoch + 1}/{num_epochs}], Step [{i + 1}/{len(train_loader)}], Loss: {loss.item():.4f}")

    # Calculate and print accuracy
    accuracy = 100 * total_correct / total_samples
    print(f"Epoch [{epoch + 1}/{num_epochs}], Training Accuracy: {accuracy:.2f}%")

    # Calculate and store average loss and accuracy for the epoch
    average_loss = total_loss / len(train_loader)
    losses.append(average_loss)
    accuracies.append(accuracy)

print("Training finished.")

I have just added two new lists and appended the loss and accuracy of each epoch to plot them at the end of this training loop. so, if we run the cell, the training loop starts.

Epoch [1/10], Step [100/782], Loss: 1.7972
Epoch [1/10], Step [200/782], Loss: 1.6378
Epoch [1/10], Step [300/782], Loss: 1.8636
Epoch [1/10], Step [400/782], Loss: 1.7561
Epoch [1/10], Step [500/782], Loss: 1.5374
Epoch [1/10], Step [600/782], Loss: 1.3903
Epoch [1/10], Step [700/782], Loss: 1.9298
Epoch [1/10], Training Accuracy: 37.66%
Epoch [2/10], Step [100/782], Loss: 1.4052
Epoch [2/10], Step [200/782], Loss: 1.5323
Epoch [2/10], Step [300/782], Loss: 1.4231
Epoch [2/10], Step [400/782], Loss: 1.4050
Epoch [2/10], Step [500/782], Loss: 1.6354
Epoch [2/10], Step [600/782], Loss: 1.5001
Epoch [2/10], Step [700/782], Loss: 1.4958
Epoch [2/10], Training Accuracy: 47.43%
Epoch [3/10], Step [100/782], Loss: 1.4195
Epoch [3/10], Step [200/782], Loss: 1.4136
Epoch [3/10], Step [300/782], Loss: 1.3760
Epoch [3/10], Step [400/782], Loss: 1.3983
Epoch [3/10], Step [500/782], Loss: 1.4408
Epoch [3/10], Step [600/782], Loss: 1.5327
Epoch [3/10], Step [700/782], Loss: 1.4464
Epoch [3/10], Training Accuracy: 52.01%
Epoch [4/10], Step [100/782], Loss: 0.9776
Epoch [4/10], Step [200/782], Loss: 1.3968
Epoch [4/10], Step [300/782], Loss: 1.2217
Epoch [4/10], Step [400/782], Loss: 1.2327
Epoch [4/10], Step [500/782], Loss: 1.2769
Epoch [4/10], Step [600/782], Loss: 1.2474
Epoch [4/10], Step [700/782], Loss: 1.2011
Epoch [4/10], Training Accuracy: 55.39%
Epoch [5/10], Step [100/782], Loss: 1.4723
Epoch [5/10], Step [200/782], Loss: 1.3214
Epoch [5/10], Step [300/782], Loss: 1.2499
Epoch [5/10], Step [400/782], Loss: 1.0077
Epoch [5/10], Step [500/782], Loss: 1.2315
Epoch [5/10], Step [600/782], Loss: 1.3217
Epoch [5/10], Step [700/782], Loss: 0.9848
Epoch [5/10], Training Accuracy: 58.57%
Epoch [6/10], Step [100/782], Loss: 0.9484
Epoch [6/10], Step [200/782], Loss: 1.0160
Epoch [6/10], Step [300/782], Loss: 0.9097
Epoch [6/10], Step [400/782], Loss: 1.1121
Epoch [6/10], Step [500/782], Loss: 1.0643
Epoch [6/10], Step [600/782], Loss: 1.1222
Epoch [6/10], Step [700/782], Loss: 0.9151
Epoch [6/10], Training Accuracy: 61.42%
Epoch [7/10], Step [100/782], Loss: 1.0603
Epoch [7/10], Step [200/782], Loss: 0.9580
Epoch [7/10], Step [300/782], Loss: 1.1307
Epoch [7/10], Step [400/782], Loss: 0.9121
Epoch [7/10], Step [500/782], Loss: 0.8515
Epoch [7/10], Step [600/782], Loss: 0.8278
Epoch [7/10], Step [700/782], Loss: 0.9549
Epoch [7/10], Training Accuracy: 64.02%
Epoch [8/10], Step [100/782], Loss: 0.7921
Epoch [8/10], Step [200/782], Loss: 0.9545
Epoch [8/10], Step [300/782], Loss: 0.7991
Epoch [8/10], Step [400/782], Loss: 1.0606
Epoch [8/10], Step [500/782], Loss: 1.2592
Epoch [8/10], Step [600/782], Loss: 0.9369
Epoch [8/10], Step [700/782], Loss: 0.9270
Epoch [8/10], Training Accuracy: 67.04%
Epoch [9/10], Step [100/782], Loss: 0.7501
Epoch [9/10], Step [200/782], Loss: 1.1043
Epoch [9/10], Step [300/782], Loss: 0.8589
Epoch [9/10], Step [400/782], Loss: 1.1142
Epoch [9/10], Step [500/782], Loss: 0.7783
Epoch [9/10], Step [600/782], Loss: 0.9715
Epoch [9/10], Step [700/782], Loss: 0.8937
Epoch [9/10], Training Accuracy: 69.46%
Epoch [10/10], Step [100/782], Loss: 0.5709
Epoch [10/10], Step [200/782], Loss: 0.5713
Epoch [10/10], Step [300/782], Loss: 0.9997
Epoch [10/10], Step [400/782], Loss: 0.6164
Epoch [10/10], Step [500/782], Loss: 1.0069
Epoch [10/10], Step [600/782], Loss: 1.1473
Epoch [10/10], Step [700/782], Loss: 0.6924
Epoch [10/10], Training Accuracy: 71.88%
Training finished.

With the same dataset, however, with different architecture, we have been able to increase the accuracy and decrease loss.

# Save the trained model
torch.save(model.state_dict(), "advanced_ann_cifar10.pth")

In this step, we will be saving the trained model, and .pth is the extension for PyTorch model, .pt can also be used instead. We will talk about loading a saved model in upcoming tutorials and using it to predict non-trained data.

Now, let's plot a line plot for loss and accuracy.

# Plotting loss and accuracy
plt.figure(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.plot(losses, label='Loss')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(accuracies, label='Accuracy', color='orange')
plt.xlabel('Epoch')
plt.ylabel('Accuracy (%)')
plt.legend()

plt.tight_layout()
plt.show()

conclusion

In this Tutorial, I have explained how to build an advanced artificial neural network from scratch with PyTorch. We learnt that with an increase in the number of layers, the overall performance of the model can be enhanced. But let me remind you just to get better results, we will not be increasing more and more layers. There are other ways to do that and we will talk about it in the upcoming days.

Hello everyone, it's me Bibek Chalise and welcome to my Blog Series. In this tutorial, we will start a new series of tutorials and that will be Computer Vision with PyTorch. PyTorch is one of the machine learning frameworks and is based on Torch Library, and makes Computer Vision, Natural Language Processing and other machine learning and deep learning tasks easier. So, from today we will focus on using Computer Vision with PyTorch and I am assuming that you know a bit about computer vision background and PyTorch fundamentals like tensors and basic Neural Network stuff. But, if you don't know that too, we will slowly get to know about that stuff too. It's going to be a good series to cover up and finish. In this blog, we will be designing a very simple Artificial Neural Network to classify 10 different classes from very popular CIFAR-10 dataset. We will be downloading it from torchvision dataset hub, so there will be no fancy loading of our custom datasets in this particular video. But will do that for sure in upcoming videos. However, before starting the code, we need to understand a bit about how we are going to approach this task.

What is ANN?

Figure: Artificial Neural Network architectural diagram [source_image] ANNs (also called Artificial Neural Networks or simply neural networks), are computing systems vaguely inspired by the biological neural networks that learn from the data we feed them. So, in this case, since we will be feeding them images, they will learn patterns in the images and the classes associated with that pattern.

This tutorial is also available in Youtube in my youtube Channel.

This is a sample of the dataset, so with each image of airplane we feed in the network, it generates a pattern for aeroplanes like it has two wings, metal like body and any other specific, we let the network decide what pattern to learn and whenever a new image is a feed into the network, it analyzes the pattern in the new test data and aligns with the pattern it previously learnt and gives us a probability result class.

What next?

So, I think we have a basic overview of what ANN is and how it will work. But how do we give our image of size 32*32 (Height and weight) coloured (3 channels) images to the Neural Network in such a way that it generalizes a pattern? In this case, what we need is to flatten this image to a single dimension. So, the 3 * 32 * 32 image of 3 dimensions will be converted to reshaped as 3072 * 1 dimensional vector. So we will have 3072 nodes as input nodes and will be projected to a definite number of nodes in the hidden layer.

Without further due, let's dive directly into the code. We will continue to know more about the structure along with the code in the video. Also, I am assuming you are well familiar with Google Colab, since we will be using that for this tutorial.

First of all, we need to import important libraries.

import torch
import torchvision
import torchvision.transforms as transforms
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F

The torch is the main library of the PyTorch framework and torchvision is another PyTorch library for all computer vision tasks. Transforms will be used for any kind of data transformation or augmentation in the images, torch.nn will be the core Neural network class of Torch, torch.optim will be for Optimisers and torch.nn.functional has many functions that will be useful as we progress in the series.

# Define data transformations
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))  # Normalize images to [-1, 1]
])

In this cell, what we are trying to do is transforms the image to tensor, since PyTorch requires the values to be a Tensor, (like ndarray of Numpy), and ToTensor converts the image values in the range of 0 to 1. Then we normalize them with a mean 0.5 and 0.5 standard deviation for each channel, Remember we are working with RGB images, right?

# Load CIFAR-10 dataset
dataset = torchvision.datasets.CIFAR10(root='./cifar', train=True, transform=transform, download=True)
dataloader = torch.utils.data.DataLoader(dataset, batch_size=64, shuffle=True)

It's funny right, we have described how we will be going to transform our images but haven't loaded the images in our working directory. torchvision.datasets has many working datasets and we have chosen CIFER10 and we have downloaded it to ./cifer folder with parameter download=True. The train=True downloads only the train dataset from the hub whereas, transform=transform makes all the transformations to the downloaded images as described above. The Dataloader function helps us to feed inputs to the model in the batch of 64 images at one time.

class SimpleANN(nn.Module):
    def __init__(self):
        super(SimpleANN, self).__init__()
        self.fc1 = nn.Linear(3 * 32 * 32, 128)  # Flatten the 32x32 RGB images
        self.fc2 = nn.Linear(128, 64)
        self.fc3 = nn.Linear(64, 10)  # Output layer for 10 classes

    def forward(self, x):
        x = x.view(-1, 3 * 32 * 32)  # Flatten the input images
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

Now the big game time, we define our model. We make a Python class named SimpleANN and it is a subclass of nn.Module class which is the base class for all the Neural Network modules in PyTorch. The __init__(self) method called whenever SimpleANN class is called. So as I mentioned, we are converting the size of our image 3 * 32 * 32 into a one-dimensional vector and feeding as input to the ANN. Then, we connect that with 128 nodes in the first hidden layer, then that is connected to 64 nodes in the second hidden layer and finally we have 10 nodes as output layer for each of the 10 output classes.

The forward function is the forward pass to carry the input image through all the layers. In the forward layer, we have used Rectified Linear Unit or ReLU has been used which is a popular Activation Function which takes the input and outputs the input in between the maximum of 0 or input. So, this keeps non-linearity in the network to improve the performance of the network.

model = SimpleANN()
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)

Now, we need to call the instance of SimpleANN class and we created a model as its instance. Also, we defined CrossEntropyLoss is our loss function which is used to calculate the loss during each training epoch and SGD or Stochastic Gradient Descent is used as our optimizer to update the weights of our network. So, lr=0.001 is the learning rate and defines the rate at which the weights are updated. The momentum parameter is to help us accelerate convergence.

# Training loop with accuracy calculation
epochs = 10
for epoch in range(epochs):
    running_loss = 0.0
    correct = 0
    total = 0
    for i, data in enumerate(dataloader, 0):
        inputs, labels = data
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        running_loss += loss.item()

        # Calculate accuracy
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

        if i % 100 == 99:
            print(f'Epoch {epoch + 1}, Mini-batch {i + 1}, Loss: {running_loss / 100:.3f}, Accuracy: {(100 * correct / total):.2f}%')
            running_loss = 0.0

print('Finished Training')

So, now we are in the training loop. we have chosen epochs=10 and in each epoch, we are taking 64 images and their corresponding labels from the data loader. Then, we feed that image to the model which gives us an output and we are only interested in the node which has the maximum probability. So, if the output is the same as the label, we increment it corret+1 and we calculate the accuracy. So, in this loop, there are a few things to understand.

    optimizer.zero_grad()
    outputs = model(inputs)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

So, here optimizer starts with zero gradients for each epoch and only after the calculation of loss by the difference between predicted output and labels, the backward propagation starts and the optimizer updates the weights as per the loss. We zero out all of our gradients so that they don't accumulate over time and then calculate new ones using a backpropagation algorithm.

Epoch 1, Mini-batch 100, Loss: 1.306, Accuracy: 54.05%
Epoch 1, Mini-batch 200, Loss: 1.324, Accuracy: 53.85%
Epoch 1, Mini-batch 300, Loss: 1.337, Accuracy: 53.52%
Epoch 1, Mini-batch 400, Loss: 1.318, Accuracy: 53.50%
Epoch 1, Mini-batch 500, Loss: 1.312, Accuracy: 53.63%
Epoch 1, Mini-batch 600, Loss: 1.308, Accuracy: 53.85%
Epoch 1, Mini-batch 700, Loss: 1.311, Accuracy: 53.88%
Epoch 2, Mini-batch 100, Loss: 1.315, Accuracy: 54.02%
Epoch 2, Mini-batch 200, Loss: 1.276, Accuracy: 54.66%
Epoch 2, Mini-batch 300, Loss: 1.287, Accuracy: 54.74%
Epoch 2, Mini-batch 400, Loss: 1.276, Accuracy: 54.80%
Epoch 2, Mini-batch 500, Loss: 1.311, Accuracy: 54.75%
Epoch 2, Mini-batch 600, Loss: 1.268, Accuracy: 54.96%
Epoch 2, Mini-batch 700, Loss: 1.275, Accuracy: 55.04%
Epoch 3, Mini-batch 100, Loss: 1.279, Accuracy: 55.28%
Epoch 3, Mini-batch 200, Loss: 1.265, Accuracy: 55.54%
Epoch 3, Mini-batch 300, Loss: 1.270, Accuracy: 55.62%
Epoch 3, Mini-batch 400, Loss: 1.259, Accuracy: 55.79%
Epoch 3, Mini-batch 500, Loss: 1.263, Accuracy: 55.89%
Epoch 3, Mini-batch 600, Loss: 1.248, Accuracy: 56.00%
Epoch 3, Mini-batch 700, Loss: 1.259, Accuracy: 55.91%
Epoch 4, Mini-batch 100, Loss: 1.223, Accuracy: 56.84%
Epoch 4, Mini-batch 200, Loss: 1.223, Accuracy: 57.07%
Epoch 4, Mini-batch 300, Loss: 1.229, Accuracy: 57.10%
Epoch 4, Mini-batch 400, Loss: 1.253, Accuracy: 56.86%
Epoch 4, Mini-batch 500, Loss: 1.218, Accuracy: 57.06%
Epoch 4, Mini-batch 600, Loss: 1.235, Accuracy: 56.99%
Epoch 4, Mini-batch 700, Loss: 1.252, Accuracy: 56.89%
Epoch 5, Mini-batch 100, Loss: 1.177, Accuracy: 58.47%
Epoch 5, Mini-batch 200, Loss: 1.215, Accuracy: 57.97%
Epoch 5, Mini-batch 300, Loss: 1.214, Accuracy: 58.03%
Epoch 5, Mini-batch 400, Loss: 1.162, Accuracy: 58.38%
Epoch 5, Mini-batch 500, Loss: 1.206, Accuracy: 58.28%
Epoch 5, Mini-batch 600, Loss: 1.215, Accuracy: 58.09%
Epoch 5, Mini-batch 700, Loss: 1.239, Accuracy: 57.92%
Epoch 6, Mini-batch 100, Loss: 1.188, Accuracy: 58.33%
Epoch 6, Mini-batch 200, Loss: 1.169, Accuracy: 58.79%
Epoch 6, Mini-batch 300, Loss: 1.165, Accuracy: 58.77%
Epoch 6, Mini-batch 400, Loss: 1.195, Accuracy: 58.73%
Epoch 6, Mini-batch 500, Loss: 1.184, Accuracy: 58.74%
Epoch 6, Mini-batch 600, Loss: 1.195, Accuracy: 58.55%
Epoch 6, Mini-batch 700, Loss: 1.199, Accuracy: 58.46%
Epoch 7, Mini-batch 100, Loss: 1.172, Accuracy: 59.00%
Epoch 7, Mini-batch 200, Loss: 1.157, Accuracy: 58.99%
Epoch 7, Mini-batch 300, Loss: 1.159, Accuracy: 59.08%
Epoch 7, Mini-batch 400, Loss: 1.158, Accuracy: 59.22%
Epoch 7, Mini-batch 500, Loss: 1.170, Accuracy: 59.25%
Epoch 7, Mini-batch 600, Loss: 1.174, Accuracy: 59.15%
Epoch 7, Mini-batch 700, Loss: 1.159, Accuracy: 59.12%
Epoch 8, Mini-batch 100, Loss: 1.128, Accuracy: 60.77%
Epoch 8, Mini-batch 200, Loss: 1.124, Accuracy: 61.12%
Epoch 8, Mini-batch 300, Loss: 1.133, Accuracy: 60.99%
Epoch 8, Mini-batch 400, Loss: 1.152, Accuracy: 60.70%
Epoch 8, Mini-batch 500, Loss: 1.140, Accuracy: 60.70%
Epoch 8, Mini-batch 600, Loss: 1.156, Accuracy: 60.41%
Epoch 8, Mini-batch 700, Loss: 1.172, Accuracy: 60.16%
Epoch 9, Mini-batch 100, Loss: 1.119, Accuracy: 61.19%
Epoch 9, Mini-batch 200, Loss: 1.120, Accuracy: 60.77%
Epoch 9, Mini-batch 300, Loss: 1.132, Accuracy: 60.68%
Epoch 9, Mini-batch 400, Loss: 1.106, Accuracy: 60.84%
Epoch 9, Mini-batch 500, Loss: 1.125, Accuracy: 60.79%
Epoch 9, Mini-batch 600, Loss: 1.139, Accuracy: 60.73%
Epoch 9, Mini-batch 700, Loss: 1.118, Accuracy: 60.79%
Epoch 10, Mini-batch 100, Loss: 1.088, Accuracy: 61.70%
Epoch 10, Mini-batch 200, Loss: 1.074, Accuracy: 61.97%
Epoch 10, Mini-batch 300, Loss: 1.103, Accuracy: 61.71%
Epoch 10, Mini-batch 400, Loss: 1.090, Accuracy: 61.99%
Epoch 10, Mini-batch 500, Loss: 1.121, Accuracy: 61.59%
Epoch 10, Mini-batch 600, Loss: 1.113, Accuracy: 61.45%
Epoch 10, Mini-batch 700, Loss: 1.100, Accuracy: 61.44%
Finished Training

This is the result of the training loop and at the end of 10 epochs, we achieved a loss of 1.1 and loss of 61.44%. This was the simplest Neural Network designed, so we can tweak over optimizer type, number of epochs, batch_size, and number of layers in the architecture of the ANN. In the next tutorial, we will be looking at the design of a more complex ANN for a new dataset and see how a change in architecture makes an impact on the overall performance of the model. So, until, then I sign off from this tutorial and see you again.

References

• PyTorch Tutorials

• source_image

• ColabNotebook

In the last article, we discussed why evaluating GANs model is hard and what are the ways to evaluate it. We concluded that it's better to use the feature distance method rather than the pixel distance method.

What is feature distance?

It is a method of using features extracted from the classifier to differentiate between real and fake images.

Classifier -> Features -> Differentiate between real and fake.

We extract the features of images using a pre-trained classifier. The fully connected layers are used for the classification and the CNN part is used for feature extraction. So, why not only use it and eliminate fully connected layers? The max-pooling layers give the features.

It is experimental to know which pooling layers to use because you never know which pooling layer gives the best-needed features.

The features are called embeddings as the features are condensed to smaller numbers.

Since the real and fake image embeddings are closer, so we can say, the generator is doing well.

How to compare embeddings?

Frechet Inception Distance (FID)

It can be used to calculate the distance between curves and distributions.

$$d(x,y) = (uX, -uY)^2 + (\sigma X - \sigma Y)^2$$

The mean gives you a sense of the centre and the standard deviation gives how far apart are the distribution.

Multivariate Normal Distributions

It is the normal distribution over large dimensions.

so, we use covariance as it measures the variance between two distributions.

so, for multivariate normal distribution, we use the following formula.

$$FID = ||uX - uY||^2 + Tr(\sum X + \sum Y - 2 \sqrt{ \sum X \sum y} )$$

Where:

Tr = Trace of a matrix (sum of diagonal)

Real and fake embeddings are two multivariate normal distributions.

Then we can compare real and fake with the FID formula.

X = Real

Y = Fake

$$uX = Mean. of. Real$$

$$ uY = Mean.of.fake$$

$$ \sum X = Covariance.of.Real.embeddings$$

$$ \sum Y = Covariance .of.Fake.Embeddings$$

The lower the FID, the closer the distributions, the better.

Shortcomings of FID

• Use pre-trained inception-v3 moel which may not capture all features

• Needs a large sample size

• Limited statistics used (Mean and standard deviation)

• Slow to run.

There is another method known as inception score which can be used to compare the features. (Readers are advised to study it by themselves, The math too vague to include in this blog.)

Sampling and Truncation

Sampling is an important technique. When you use random noise of mean zero and standard deviation of one, you get better quality results but they look like normal images with less variation. However, when you use another random noise you get more diverse images but of less quality. so, there is always a trade-off.

We can conclude that the evaluation of GANs model is very much sample dependent rather than model parameters, so choose wisely how you sample.

So, to reduce the trade-off between diversity and fidelity we can use the truncation technique.

Fig: Truncation of the sample.

Thank you!

A discriminator cannot be used for evaluation, because it overfits the generator it's trained with.

Two important properties when it comes to generating images from GANs.

1. Fidelity

2. Diversity

Fidelity deals with:

-> How good are the fake images?

-> How far is the fake image from the real one?

Diversity deals with:

You want a GAN that can generate a variety of different images.

Comparing images

There are two ways to compare real and fake images generated by Generator.

1. Pixel Distance

2. Feature Distance

Pixel Distance

Fig: Real Image - Fake Image = Abs. distance

This is not a great measure, if a single pixel is shifted, the abs. the difference will be significantly changed.

Feature Distance

Fig: Feature Distance

With this technique, even a shift in pixel values won't make any difference.

Conclusion:

-> Pixel distance is simple but unreliable.

->Feature distance uses the higher-level features of an image, making it more reliable.

The following two images are generated by Deep Convolution Generative Adversarial Network trained over Devanagari Numbers and Letters.

Fig: generated images

The dataset was taken from GitHub (https://github.com/kcnishan/Nepali_handwritten_digits_recognition) and the framework used to design this GAN was PyTorch.

Fig: Original Dataset

Then, the image path and its label are converted to dataframe using the following script.

img_path = []
label = []
for each in os.listdir(train_path):
  for x in os.listdir(train_path+'/'+each):
    img_path.append(train_path+'/'+each+'/'+x)
    label.append(each)

img_path_test = []
label_test = []
for each in os.listdir(Test_path):
  for x in os.listdir(Test_path+'/'+each):
    img_path_test.append(Test_path+'/'+each+'/'+x)
    label_test.append(each)

df = pd.DataFrame(list(zip(img_path, label)), columns=['path', 'label'])
df_test = pd.DataFrame(list(zip(img_path_test, label_test)), columns=['path', 'label'])
LbE = LabelEncoder()
df.label =  LbE.fit_transform(df.label)
df_test.label =  LbE.fit_transform(df_test.label)

Then a custom dataset loader was designed which returns tensor values of images and target label. The label was not used for GAN but later was useful for prediction.

class LetterDataset(Dataset):
  def __init__(self, df):
    self.fpath = df.path
    self.label = df.label
    self.transform = T.Compose(
   [T.ToPILImage(),
    T.ToTensor(),  
    T.Normalize(
       mean=[0.485, 0.456, 0.406],
       std=[0.229, 0.224, 0.225]
   ),
    T.Grayscale(num_output_channels=1)
    ]
)

  def __len__(self):
    return len(self.fpath)

  def __getitem__(self, ix):
    img = self.fpath[ix]
    target = self.label[ix]
    img = cv2.imread(img)
    im = self.transform(img)
    return torch.tensor(im/255).to(device).float(), torch.tensor(target).long().to(device)

BCELoss is used as the loss algorithm and Adam as the optimiser. As per the beta, the optimiser parameters such as beta1, and beta2 values are taken as 0.5,0.999 and the learning rate as 0.0002. The models are trained at 50 epochs.

image_size = 32
batch_size = 32
z_dim =100
beta1 = 0.5
lr=0.0002
epochs = 50
sample_size=8

The initial size of images is 32 * 32 and no changes are done with the image height and width but all images are normalised to ( mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] )

(
       mean=[0.485, 0.456, 0.406],
       std=[0.229, 0.224, 0.225]
   )

The basis GAN architecture is:

def weights_init(m):
               classname = m.__class__.__name__
               if classname.find('Conv') != -1:
                   nn.init.normal_(m.weight.data, 0.0, 0.02)
               elif classname.find('BatchNorm') != -1:
                   nn.init.normal_(m.weight.data, 1.0, 0.02)
                   nn.init.constant_(m.bias.data, 0)

class Generator(nn.Module):
  def __init__(self):
    super(Generator,self).__init__()
    self.model = nn.Sequential(
        nn.ConvTranspose2d(100,64*8,4,1,0,bias=False,),
        nn.BatchNorm2d(64*8),
        nn.ReLU(True),
        nn.ConvTranspose2d(64*8,64*4,4,2,1,bias=False),
        nn.BatchNorm2d(64*4),
        nn.ReLU(True),
        nn.ConvTranspose2d( 64*4,64*2,4,2,1,bias=False),
        nn.BatchNorm2d(64*2),
        nn.ReLU(True),
        nn.ConvTranspose2d(64*2,1,4,2,1,bias=False),
        nn.Tanh()
        )
    self.apply(weights_init)
  def forward(self,input): return self.model(input)

class Discriminator(nn.Module):
  def __init__(self ):
    super(Discriminator, self).__init__()
    self.disc  = nn.Sequential(

        #input size=32*32
        nn.Conv2d(1, 64, kernel_size=4, stride=2, padding=1),
          #16*16
        nn.LeakyReLU(0.2),
        disc_layer(64, 64*2, 4,2,1), #8x8
        disc_layer(64*2, 64*4, 4,2,1),#4x4
        nn.Conv2d(64*4, 1, kernel_size=4,stride=2,padding=0),#1x1
        nn.Sigmoid()
    )

    self.apply(weights_init)
  def forward(self, input):
    return self.disc(input)

The training script for both Generator and Discriminator is:

def train_disc(optimizer,real_data, fake_data):
               optimizer.zero_grad()
               prediction_real = disc(real_data)
               error_real = loss_fn(prediction_real.squeeze(), \\
                                 torch.ones(len(real_data)).to(device))
               error_real.backward()
               prediction_fake = disc(fake_data)
               error_fake = loss_fn(prediction_fake.squeeze(), \\
                                 torch.zeros(len(fake_data)).to(device))
               error_fake.backward()
               optimizer.step()
               return error_real + error_fake

def train_gen(optimizer,fake_data):
               optimizer.zero_grad()
               prediction = disc(fake_data)
               error = loss_fn(prediction.squeeze(), \\
                            torch.ones(len(fake_data)).to(device))
               error.backward()
               optimizer.step()
               return error

for i in range(50):
  loss_g = 0.0
  loss_d = 0.0
  print(f'Epoch: _________*****{i}*****_______')
  for ix, batch in tqdm.tqdm(enumerate((trn_ldr))):
    x, y = batch
    x = x.to(device)
    b_size = len(x)
    data_fake = gen(create_noise(b_size, z_dim)).detach()
    data_real = x
    loss_d +=train_disc(optimizer_d, data_real, data_fake)
    data_fake = gen(create_noise(b_size, z_dim))
    loss_g += train_gen(optimizer_g,data_fake)
  generated_image = gen(noise).cpu().detach()
  plt.imshow(generated_image[0].reshape(32,32))
  plt.show()
  print(loss_g, loss_d)
  epoch_loss_g = (loss_g / ix)
  epoch_loss_d = loss_d / ix
  losses_g.append(epoch_loss_g)
  losses_d.append(epoch_loss_d)
  print(f'Epoch: _________*****{i} Generator Training Loss : {epoch_loss_g} *****_______')
  print(f'Epoch: _________*****{i} Discriminator Training Loss : {epoch_loss_d} *****_______')

Along with generation, I have also made a CNN model which predicts the letters and consonants. The accuracy is set as a metric to determine the model worthiness and CrossEntropyLoss is the loss metric.

def conv_layer(ni, no, kernel_size, max_pool_num=2, stride=1,):
  return nn.Sequential(
      nn.Conv2d(ni, no, kernel_size, stride),
      nn.BatchNorm2d(no),
      nn.LeakyReLU(0.2),
      nn.MaxPool2d(max_pool_num)
  )

from torch.nn.modules.activation import Softmax
def get_model():
  model = nn.Sequential(
      conv_layer(1,8,2),
      conv_layer(8,16, 2),
      conv_layer(16, 64, 2),
      nn.Flatten(),
      nn.Linear(64*3*3,192),
      nn.ReLU(),
      nn.Linear(192, 64),
      nn.ReLU(),
      nn.Linear(64, 36),
      nn.Softmax()
  ).to(device)
  loss_fn = nn.CrossEntropyLoss()
  optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
  return model, loss_fn, optimizer

One Fun Fact:

In the dataset, there was a problem, a folder was inside the image folder, which was not usual and manual looking at all datasets was not possible. So, I made a script to check the faulty folder and removed it. Took a little time to do but in the end, it was an interesting task to do. After all, when you have the power of python, it becomes easy.

import os
from os import listdir
from PIL import Image

count= 0
for each in os.listdir(Test_path):
  for x in os.listdir(Test_path+'/'+each):
    if x.endswith('.png'):
      try:
        img = Image.open(Test_path+'/'+each+'/'+x)
        img.verify()
      except:
        print(each+'/'+x)
    else:
      print(each+'/'+x)

Future works:

→ Go for transfer learning in the prediction model.

→ Build better Discriminator and Generator models.

→ Try conditional GAN

→ Build an interface where people can generate images as well as classify that generated images.

The repository for the full project is:

https://github.com/bibekebib/Nepali-Letters-and-Numbers-classification-generation

Scaling a dataset is one of the major step of Data Pre-Processing, It is done to reduce the range of data variables. When it comes to image, the minimum - maximum possible value range is always 0-255, that means 255 is the maximum value. So the best way to scale down image array value is to divide it by the maximum value. So the range will always be in between 0-1.

Scaling down the input variables will keep in such a range that, it will be easier for the model to work with, will take less time.

Here, for the given example we have taken Fashion MNIST dataset described in torch.datasets module.

A brief about the dataset, it has 10 classes of wear-items, with total of 60000 data points equally distributed as 6000 each class. So, the dataset is balanced, we dont have to look for that.

First we have defined the program to train, the dataset without scaling down the dataset. More than Code Structure, We are more interested to know the difference scaling of dataset that can come.

I have in short commented out the code structure too.

# Importing all required libraries

from torch.utils.data import Dataset, DataLoader
import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
device = 'cuda' if torch.cuda.is_available() else 'cpu'
from torchvision import datasets

# Here we downloaded the datset to data_folder, with train=True indicating, it is for training purpose. 
data_folder = '/data/'

fmnist = datasets.FashionMNIST(data_folder, download=True, train=True) 
tr_images = fmnist.data
tr_targets = fmnist.targets


# Here, we have extended Dataset Class to define dataset the way we wanted, More on this on upcoming tutorials


class FMNISTDataset (Dataset):
  def __init__(self, x, y):
    x = x.float()
    x = x.view(-1, 28*28) #Flattern the input 28*28 image
    self.x, self.y = x, y
  def __getitem__(self, ix):
    x, y = self.x[ix], self.y[ix]
    return x.to(device), y.to(device)

  def __len__(self):
    return len(self.x)

# Here we simple, loaded dataset defined above, using DataLoader Module in batch_size of 32.

def get_data():
  train = FMNISTDataset(tr_images, tr_targets)
  trn_dl = DataLoader(train, batch_size=32, shuffle=True)
  return trn_dl


#   Defining the model

from torch.optim import SGD
def get_model():
  model = nn.Sequential(
      nn.Linear(28*28,1000),
      nn.ReLU(),
      nn.Linear(1000,10)
  ).to(device)

  loss_fn = nn.CrossEntropyLoss()
  optimizer = SGD(model.parameters(), lr = 1e-2)
  return model, loss_fn, optimizer

#Training the data in defined model

def train_batch(x, y, model, opt, loss_fn):
  model.train()
  prediction = model(x)
  batch_loss = loss_fn(prediction, y)
  batch_loss.backward()
  opt.step()
  opt.zero_grad()
  return batch_loss.item()

#For calculating accuracy, here @torch.no_grad() used to define that, we dont calculate gradient while testing.

@torch.no_grad()
def accuracy(x, y, model):
  model.eval()
  prediction = model(x)
  max_values, argmaxes = prediction.max(-1)
  is_correct = argmaxes ==y
  return is_correct.cpu().numpy().tolist()

#Running Model for training and testing in number of epochs


train_dl = get_data()
model, loss_fn, optimizer = get_model()
losses, accuricies = [], []
for epoch in range(5):
  print(epoch)
  epoch_losses, epoch_accuricies = [], []
  for ix, batch in enumerate(iter(train_dl)):
    x, y = batch
    batch_loss = train_batch(x, y, model, optimizer, loss_fn)
    epoch_losses.append(batch_loss)
  epoch_loss = np.array(epoch_losses).mean()
  for ix, batch in enumerate(iter(train_dl)):
    x, y  =batch
    is_correct = accuracy(x, y, model)
    epoch_accuricies.extend(is_correct)
  epoch_accuracy = np.mean(epoch_accuricies)
  losses.append(epoch_loss)
  accuricies.append(epoch_accuracy)

#For Plotting purpose. 

epochs = np.arange(5)+1
plt.figure(figsize=(20,5))
plt.subplot(121)
plt.title('Loss value over increasing epochs')
plt.plot(epochs, losses, label='Training Loss')
plt.legend()
plt.subplot(122)
plt.title('Accuracy value over increasing epochs')
plt.plot(epochs, accuricies, label='Training Accuracy')
plt.gca().set_yticklabels(['{:.0f}%'.format(x*100) \
 for x in plt.gca().get_yticks()])
plt.legend()

Here, in above plot we can see that the loss has reduced to a point where it has saturated but, the accuracy seems to be just around 13%, that's not what we want our model to perform right. So, we tweak just single hyperparameter, scaling down the dataset range. Okay, I agree, you don't want to call it hyperparameter tuning, we don't call it so.

As mentioned above, we divide the input by it's maximum possible value, i.e. 255.

The changed code will be

class FMNISTDataset (Dataset):
  def __init__(self, x, y):

    #The changed code starts here
    x = x.float()/255
    #The changed code ends here

    x = x.view(-1, 28*28)
    self.x, self.y = x, y
  def __getitem__(self, ix):
    x, y = self.x[ix], self.y[ix]
    return x.to(device), y.to(device)

Now, lets Rerun the code and see how much change it makes.

from torch.utils.data import Dataset, DataLoader
import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
device = 'cuda' if torch.cuda.is_available() else 'cpu'
from torchvision import datasets


data_folder = '/data/'

fmnist = datasets.FashionMNIST(data_folder, download=True, train=True)
tr_images = fmnist.data
tr_targets = fmnist.targets


class FMNISTDataset (Dataset):
  def __init__(self, x, y):

    #The changed code starts here
    x = x.float()/255
    #The changed code ends here

    x = x.view(-1, 28*28)
    self.x, self.y = x, y
  def __getitem__(self, ix):
    x, y = self.x[ix], self.y[ix]
    return x.to(device), y.to(device)

  def __len__(self):
    return len(self.x)



def get_data():
  train = FMNISTDataset(tr_images, tr_targets)
  trn_dl = DataLoader(train, batch_size=32, shuffle=True)
  return trn_dl



from torch.optim import SGD
def get_model():
  model = nn.Sequential(
      nn.Linear(28*28,1000),
      nn.ReLU(),
      nn.Linear(1000,10)
  ).to(device)

  loss_fn = nn.CrossEntropyLoss()
  optimizer = SGD(model.parameters(), lr = 1e-2)
  return model, loss_fn, optimizer


def train_batch(x, y, model, opt, loss_fn):
  model.train()
  prediction = model(x)
  batch_loss = loss_fn(prediction, y)
  batch_loss.backward()
  opt.step()
  opt.zero_grad()
  return batch_loss.item()


@torch.no_grad()
def accuracy(x, y, model):
  model.eval()
  prediction = model(x)
  max_values, argmaxes = prediction.max(-1)
  is_correct = argmaxes ==y
  return is_correct.cpu().numpy().tolist()



train_dl = get_data()
model, loss_fn, optimizer = get_model()
losses, accuricies = [], []
for epoch in range(5):
  print(epoch)
  epoch_losses, epoch_accuricies = [], []
  for ix, batch in enumerate(iter(train_dl)):
    x, y = batch
    batch_loss = train_batch(x, y, model, optimizer, loss_fn)
    epoch_losses.append(batch_loss)
  epoch_loss = np.array(epoch_losses).mean()
  for ix, batch in enumerate(iter(train_dl)):
    x, y  =batch
    is_correct = accuracy(x, y, model)
    epoch_accuricies.extend(is_correct)
  epoch_accuracy = np.mean(epoch_accuricies)
  losses.append(epoch_loss)
  accuricies.append(epoch_accuracy)



epochs = np.arange(5)+1
plt.figure(figsize=(20,5))
plt.subplot(121)
plt.title('Loss value over increasing epochs')
plt.plot(epochs, losses, label='Training Loss')
plt.legend()
plt.subplot(122)
plt.title('Accuracy value over increasing epochs')
plt.plot(epochs, accuricies, label='Training Accuracy')
plt.gca().set_yticklabels(['{:.0f}%'.format(x*100) \
 for x in plt.gca().get_yticks()])
plt.legend()

Wow, look at the accuracy, its around 85%, from 13% to 85% just by scaling down the input.

But what could be the reason for such drastic increase in accuracy merely scaling down the input. Lets, see the math behind this.

We know,

sigmoid =

source: Modern Computer Vision with PyTorch

In the left hand side, when the input is 255 and the weight is more than or equal to 0.1, there is no change in the sigmoid output. Similarly the change was also so less when the weight was extremely low.

The reason is that exponential of the large negative values is very close to 0. But in the right hand side, since the input is 1, we can see change in the sigmoid output.

Scaling the input dataset so that it contains a much smaller range of values generally helps in achieving better model accuracy.

GitHub

First Lets import all the requirements that is needed for building the basic architecture of Neural Network in PyTorch. If you haven't yet installed PyTorch, i strongly suggest you to install it from here (PyTorch Official Site) cause I wont be teaching it in details.

Plus, we won't talking much about PyTorch fundamentals, like tensors and other operations, but if you want it, I will make separate Tutorial on that. But in this series, I assume you are pretty familiar with basic PyTorch , numpy and python and we will continue with that assumptions. So much to cover, lets see how far we can go on this.

The first task is to import the libraries we need for the overall task, PyTorch is the must at first and if you need something else in the future, we will import requirements in subsequent cells.

# importing libraries
import torch

I have taken a toy dataset for less complications, In future we may take a real life dataset but for now lets stick with this one.

x  = [[1,2],[3,4],[5,6],[7,8]]
y = [ [3],[7],[11],[15]]

the next task is to convert it into tensor which is the building block of PyTorch library, its like numpy ndarray, but not exactly the same.

x = torch.tensor(x).float()
y = torch.tensor(y).float()

Like i mentioned above, tensor and numpy ndarray are same, but different, the difference can be seen when it comes to execution. PyTorch tensors can be executed in GPU while numpy array doesn't support execution in GPU, plus because of threading, it takes less time for PyTorch tensors to execute even in CPU than to numpy array. So, if we have GPU available, we will be using it in full extend.

Its difficult to afford GPU by us, so we will be using Free GOOGLE COLAB and enable GPU there.

from torch.cuda import is_available
device = 'cuda' if torch.cuda.is_available() else 'cpu'

print(device)

The default in Colab is CPU, so if you want to change to GPU, navigate through Runtime and change the runtime type to GPU.

X = x.to(device)
Y = y.to(device)

Now, we are into the basis foundation of Neural Network. Lets learn each step, with explanation of each step.

import torch.nn as nn 
# torch.nn is the class where everything about neural network resides.

class Nnet(nn.Module):
  #inheriting the class torch.nn.Module into Nnet It is compulsory to inherit from nn.Module
  # as it is the base class for all NN.
  def __init__(self): #making all the initializations of all components nn.Module
    super().__init__()
    # super()__init__() make sure that the class completely inherit nn.Module
    #with this, we can completely take advantage of pre-built functionlaties of nn.Module


    #define Layers in the Neural Network

    self.input_to_hidden_layer = nn.Linear(2,8)
    self.hidden_layer_activation = nn.ReLU()
    self.hidden_to_output = nn.Linear(8,1)


    #defining Forward Propgation

  def forward (self, x):
      x = self.input_to_hidden_layer(x)
      x = self.hidden_layer_activation(x)
      x = self.hidden_to_output(x)
      return x

here we are defining 3 layers, input layer, hidden layer and output layer, with activation in the hidden layer.

If we look closely, we can see we have used nn.Linear(2,8) and nn.Linear(8,1) it means, the first parameter is the number of input features to the node and the second is the number of output features from the node. This means, as in our dataset, we will be sending 2 features into the node and it will output 8 output features in the hidden layer.

The hidden layer also comes up with activation layers, in brief the activation layers makes sure either to fire or not to the node.

Here, we have used ReLU activation function, which stands for Rectified Linear Unit, The other popular activation function are

1. sigmoid
2. SoftMax
3. Tanh

The forward function is for defining forward propogation, the name "FORWARD" is compulosory as it is reserved word to define forward propagation. With other name, it would create error.

Lets create an instance of NNet class as mynet.

Also, we will look how the randomly initialized weights would look like.

mynet = Nnet().to(device)
# taking everything to the device is compulory if we want to utilize GPU.

mynet.input_to_hidden_layer.weight

image.png

Note, everytime you run the above code, the weights initialized will be different, if you want to have same, you have to specify the seed using manual seed method in torch as torch.manual_seed(42).

Now, lets define the loss function for our model. We will be using mean square loss in our case, the other available prominent loss functions can be

• CrossEntropyLoss (for multinomial classifications)
• BCELoss (Binary cross entropy loss for binary classification) But more on these in upcming tutorials.

loss_func = nn.MSELoss()

model_output = mynet(X)
loss_value = loss_func(model_output, Y)
print(loss_value)

In pytorch, for loss function, the first parameter is the predicted output and the second parameter is the actual output required.

Now, its time to optimize the model using optimizer that tries to reduce the loss value. The inputs to the optimizer will be weights, biases and learning rate when updating the weights.

Here, we will be employing Stochastic gradient descent (SGD), other optimizers will be used for other use cases.

from torch.optim import SGD
opt = SGD(mynet.parameters(), lr=0.001)

Now we need to perform the following steps in a single epoch together and run all the steps for number of loops.

• Calulate loss values correponging to given input and output
• calculate the gradient correponding to each parameter
• update the weights based on learning rate and gardient
• flush out previous epochs gardient

loss_history = []
for _ in range(50):
  opt.zero_grad() # flush out previous epochs gradients
  loss_value = loss_func(mynet(X), Y) #calculating loss value
  loss_value.backward() #performing back propagation
  opt.step() #update weights according to the gradients calculated
  loss_history.append(loss_value.cpu().detach().numpy()) 
  #The last step to convert all the tensors in GPU to cpu and then to numpy since numpy doesnt support GPU.

Lets plot out result.

import matplotlib.pyplot as plt
%matplotlib inline
plt.plot(loss_history)
plt.title('loss variation over increasing epochs')
plt.xlabel('epochs')
plt.ylabel('loss value')

Saving and loading the pytorch model

#saving model
torch.save(mynet.state_dict(), 'mymodel.pth')

#loading model
mynet.load_state_dict(torch.load('mymodel.pth'))

GitHub Link

Today lets, see how we can represent text data of a corpus in array format. As we know, computers only understand numbers, and when we are performing any machine learning algorithm, we have to encode each data into some sort of numerical format, so that the algorithms can find a pattern from that data and build a model. And if we are into Natural Language Processing and especially text-data analysis, we have to deal with the text as data. so, in order to feed to the algorithm, it is a must-performed step that we, change the textual raw data into numerical data. There are various ways to do it. Let's discuss those. The first is Bag of Words, it is just a way of counting the numbers of each text that appears in a corpus. (Here, Corpus means the entire dataset of text.) Let's take 3 sentences.

1. "It is going to rain today"
2. "I am going to drink coffee"
3. "I am going to capital today"

If we perform Bag of words in the above example, first we make count the number of times individual items, repeats in a corpus.

Now if we represent it in the tabular form, the bag of words representation looks like this.

But we can already see the problem in this Bag of Words representation, All the words carry the same importance. In the given dataset, the word 'going' is present in each of the sentences. While, words like rain, coffee, capital are present only in each sentence, and carry the main essence of the sentence. But when we represent it in the BoW model, these all words got the value of 1. So, BoW model representation, will not represent the importance of some words which can be problematic during The problem we can see is it, no order is maintained, which means the semantic information is not preserved. We know, the text is sequential data, so the order of data is very important, but the BoW model doesn't care about the order of data. So, this can cause problems when we have to work on models where data need to be in proper order so that machines can learn from the data. If you want to perform Bag of Words in python sklearn, we can perform it as.

    from sklearn.feature_extraction.text import CountVectorizer
    import pandas as pd
    vectorizer = CountVectorizer()
    doc = ["It is going to rain today",
        "I am going to drink coffee",
        "I am going to capital today"]
    X = vectorizer.fit_transform(doc)
    column = vectorizer.get_feature_names()
    df = pd.DataFrame(X.toarray(), columns=column)
    df

In order to solve the problems with the Bag of Words Model, we use something called TF-IDF. So what is TF-IDF? Tf-IDF stands for Term Frequency - Inverse Document Frequency. Here, Term Frequency means the ratio of Number of Occuracnies of a word in a Document to the Number of Words in that Document. Term frequency, tf(t,d), is the frequency of term t, where f__t,d is the raw count of a term in a document, i.e., the number of times that term t occurs in document d. There are various other ways to define term frequency.

From the above example, the term-frequency of the word going is: Here, going appears 3 times in the document and there are total 18 words. so, tf(going) = 3/18 = 0.1666 similarly, the tf of word to is : tf(to) = 2/18 = 0.111

so, let calculate the term frequency for all the terms:

Since we have calculated the term-frequency, let's discuss Inverse Document Frequency (IDF). IDF is calculated as the log of the ratio of Numbers of the document to the Number of documents that contain the particular term. So, measure the amount of value the word provides i.e, is the measurement of how common or how rare is the word in the given corpus. with

• : total number of documents in the corpus N = | D |
• : number of documents where the term t appears (i.e., t f ( t , d ) ≠ 0 ). If the term is not in the corpus, this will lead to a division-by-zero. It is therefore common to adjust the denominator to 1 + | { d ∈ D: t ∈ d }

So, let's calculate the IDF value of some terms. The IDF of 'going' can be calculated as: Word 'going' is present in all three documents and there are since total 3 documents. so the idf value of going must be, idf(going) = log(3/)= log(1) = 0. What it tells that since going is present in all the 3 documents, it carries no importance at all. Also, if we calculate the idf value of to, it becomes: idf(to) = log(3/2) = 0.17609 Also, if we calculate the idf value of coffee, it becomes: idf(coffee) = log(3/1) = 0.47712 So, let's see what IDF value of each term becomes.

Now, it's time to do magic, calculate TF-IDF. It is simply the product of Term Frequency and Inverse Document Frequency. If we calculate the TF-IDF value of the word to in document 1, we get. TFIDF(to) = TF(to) IDF(to) = 0.16660.17609

This is the final TF-IDF text representation for the example corpus. You can try TF-IDF in sklearn as given below code.

    from sklearn.feature_extraction.text import TfidfVectorizer
    vectorizer = TfidfVectorizer()
    X = vectorizer.fit_transform(doc)
    column = vectorizer.get_feature_names()
    df = pd.DataFrame(X.toarray(), columns=column)

If you have tried TF-IDF in sklearn, then you can see that the results are quite different. It is because the sklearn TI-IDF vectorizer uses the log normalization method for the calculation and has tuned parameters in a different way. The above-mentioned method is the root idea about TFIDF, yet it needs to be tuned for large extensive use.

If you are still confused in TFIDF, let me know in the comments, until then, enjoy Learning. The code for this tutorial can also be found at this link.

Thank you!

What are Hyperparameters actually?

Let’s see this way when using a machine learning algorithm, there are various parameters associated with the instance or the method we using of a particular algorithm By default, it is provided, which gives significantly good results. However, if we want to increase the accuracy of the results, we have to make some tweaks to the default parameters. And the process of tuning such parameters with the hope of better accuracy of the given model using a particular algorithm instance can be called Hyperparameter Tuning. If it looks like Jargon, we will look at an example of the default parameter of the Support Vector Machine Classifier SVC instance.

In the above example, when we see the parameters of the SVC instance, we get the default parameters as mentioned above. So when we instantiate the SVC instance, the default parameters are passed in it. But when we visit the official documentation of SVC, we see a bunch of these parameters can be passed as a dictionary or list. So, we use that feature of such flexibility of these parameters and try a different set of parameters and find the best parameters that give the best results.

So, what we do is take a dataset and work on it and find the accuracy by default parameters and then tune few parameters to increase the score.

For this task, we will be using Jupyter Notebook. If you like doing it in a local machine it's okay, but I highly suggest using the online Jupyter Notebook. Colab by Google is a very good resource that we can use for free and Deepnote is another alternative to Google Colab. Here, I personally will be using Deepnote.

    #importing required moduls.
    import pandas as pd #for tabular data frame analysis
    import numpy as np. #Form mathematical Manipulation
    import matplotlib.pyplot as plt #for data visulaization
    import seaborn as sns #Seaborn is developed on top of matplotlib library

So, we need a dataset for it. There are various datasets available in kaggle. And we take a simple dataset from Kaggle Heart Disease Dataset.

#loading dataset
df = pd.read_csv('./heart.csv') #The dataset is downloaded and saved to root folder.
df.head()

After this, we get the first five rows of the dataset.

However, if we look closely to Target feature, we see all 1's and if we make even close oversvation with df['target'], we see a pattern that first half of the dataset has 1 value in target feature and remaining has 0. This can be a great problem and can result to bad in the training, testing phase. So, what we do is shuffle this dataset using pandas.sample() method.

    df = df.sample(frac = 2, random_state=42, replace = True)
    df.head()

After this, when we analyse the dataset, we see random distribution of 0's and 1's in the target variable. Now, what we do is see if there are any null or missing value in the dataset.

    df.isnull().sum()

Here, we can see that, there are no null values.

Now we are set to go for machine Learning Tasks

First we import required modules.

    #Import Machine Learning Libraries
    from sklearn.model_selection import train_test_split
    from sklearn import svm 
    from sklearn.model_selection import RandomizedSearchCV

The required modeules are imported. train_test_split is for dividing the dataset into training and testing sub-dataset. The svm is Support Vector Machine Algorithm. The RandomizedSearchCV is for hyperparameter Tuning. Alternative to RandomizedSearchCV is GridSearchCV, however RandomizedSearchCV is likely to be faster than GridSearchCV.

    X = df.drop(['target'], 1)
    y = df.target
    print(X.shape, y.shape)

Then we created a dataframe X which consists of Feature Variables, target is dropped because it is not Feature variable, rather it is target variable. y is defined as pandas series object with target as it only feature.

    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
    clf = svm.SVC()
    clf.fit(X_train, y_train)
    print("The accuracy of the classifier is {}".format(clf.score(X_test, y_test)))

In this step, we divided X, y into train and test sub-dataset. train_test_split returns four objects, so we stored those values into X_train, X_test, y_train, y_test. The parameters are X, y, and the test_size=0.2 parameters defines what percentage of dataset is to be described for test_set which in this case are X_test and y_test. then we instantiated SVC (Support vector Classifier) into vairable clf and used fit() method to fit, X_train and y_train. The accuracy of the classifier is found to be mere 68.85%.

Sadly, 68.85% percentage accuracy is very less, so we try to tune certain parameters and improve the accuracy.

#Lets try tuning some hyperparameters.
    param_dist = {'C': [0.1, 1, 10, 100, 1000],
    'gamma': [1, 0.1, 0.01, 0.001, 0.0001],
    'kernel': ['rbf']
    }
    svc_hyper = RandomizedSearchCV(SVC(), param_distributions=param_dist, verbose=2, cv=3, random_state=42, n_iter=10, scoring='accuracy')
    svc_hyper.fit(X_train, y_train)

Here, we used different set of parameters like C, gamma and kernel to loop through set of combinations of prameters and at the end define which set of combination of these parameters gives the best result. Here C is Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. The penalty is a squared l2 penalty. The parameter gamma is Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’. And the parameter kernel Specifies the kernel type to be used in the algorithm. It must be one of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable. If none is given, ‘rbf’ will be used. Here, we used only 'rbf' because other kernel takes significant time to get trained. You, yourself can try other kernels and see if that changes the results. To know more about SVC, go through this.

    svc_hyper.best_params_

we get the best parameter as {'kernel': 'rbf', 'gamma': 0.001, 'C': 1000}. So, lets use it to fit the data.

    best_svc = SVC(C=1000, gamma=0.001, kernel='rbf')
    best_svc.fit(X_train, y_train)
    print("The accuracy of the classifier is {}".format(best_svc.score(X_test, y_test)))

After fitting the data using SVC method and using the best parameter, we got the accuracy to be 94.26%. That's remarkable to what we observe at first place as 68.85%.

Hence in this way, we can use RandomizedSearchCV to tune the parameters and increase the accuracy.

GitHub Repo of the code: https://github.com/bibekebib/Hyperpramater-tuning-article-code

Deepnote Shared code: https://deepnote.com/@bibek-chalise/Hyperparameters-Tuning-Tutorial-j46REW6sTXaWqbz8APnolQ#

If you want to try this with other Algoithms, here is a list of parameters that you can hypertune.

    #Random Forest 
    n_estimator = [int(x) for (x) in np.linspace(100, 1200, num=12)]
    max_depth = [int(x) for x in np.linspace(5, 30, num=6)]     
    min_samples_split = [2, 5, 10, 15, 100]
    min_samples_leaf = [1, 2, 5, 10] criterion = ['gini', 'entropy']
    param_dist = { "n_estimators" : n_estimator, "max_depth" : max_depth, "min_samples_leaf":min_samples_leaf, "criterion":criterion, "min_samples_split":min_samples_split }

    #KNN
    n_neighbors = [int(x) for x in np.linspace(start = 1, stop = 100, num = 50)]
    weights = ['uniform','distance'] 
    metric = ['euclidean','manhattan','chebyshev','seuclidean','minkowski'] 
    random_grid = { 'n_neighbors': n_neighbors, 'weights': weights, 'metric': metric, }

    #Logistic Regression
    param_dist = { 'penalty' : ['l1', 'l2'], 
    'C' : [0, 1, 2, 3, 4] 
    }

    #Gaussian Naive
    params_NB = {'var_smoothing': np.logspace(0,-9, num=100)}

Happy Learning!

Cold outside, you all know what Pokhara serves in the mid-winter. Too cold, one blanket is never enough to comfort us. Already had switched off the light, was about to switch off my eyes too.

“Ohh! Somebody messaged me, Can’t believe my eyes.” I replied with winky emoticons.

“Pagal!”

She Fired some angry emoticons.

I called her, she received the call in a microsecond.

“Hello, angry girl.”

“I am not talking with you.”

“I heard something like this a few hours back.”

She always does this, her mood and her anger swing more than the bouncy tracks of Lords Cricket Stadium.

We had this conversation the same day a few hours back, which ended with a romantic fight.

“I will never call you, never going talk to you, ever and this ever is for true. Never.”

I just said, “Okay my angry flappy bird, see you never.”

“I hate you” she replied.

I was blushing by this side, however, Replied to her, “I hate you 3000, and always remember, I hate you the most. Don’t call me, keep it in your mind, don’t call me.”

“I hate you even more.” she continuously copy-pasted this.

“How was the day?” she looked even more beautiful when the ray of the LED light kisses so softly by her lips, the glow I see in those sparkling eyes, I wish I had words for it. The reasons we are together, her love for me, that endless love which a fight can never end. She is crazy, that craziness drives me mad each day, makes us closer each time, deep inside i miss her, she misses me even more. Different cities, the overloaded pressure on my head, her studies, our respective dreams, the small world we build each day under the canopy of respect and attachment we share, and the miracle of how we see each other life, we are stronger, just perfect regardless the imperfections we own.

“I missed my heart and finished the whole damn chapter. Bibek, I know I will crack the job, I feel it.” I felt the child, I felt the emotions in those words, a dream, a charisma in her eyes. So vulnerable she, yet so fighter, I just paralyze in those symmetrical eye-lined contours of lips and those deep eyes.

“You going to make me, us proud one day for sure.”

“Didn’t you miss me?”, She wanted to feel special, she was special but that’s she, we all want to be missed by someone, loved by someone, that someone that sees us differently, that treats us especially.

“Nah, why would I miss you, Sorry!” I said with a winky smile.

“Goodbye, Goodnight, I don’t wanna talk to you, I just wanna kill you!”

“Pagal! I hate you.”

“I know.”

Knock! knock,

“what’s up, man?”

“Water!”, “talking with vauju??”

“Yeah.”

“Was it Samir?” She asked me after my roommate returned.

“Umm, Came by to drink water.”

“Why did you choose the kitchen to sleep in?” She asked.

“We can only afford two rooms, one kitchen, and a bedroom, let’s just say we fought head to toe and I landed here.”

“Pretty bad luck my man” She laughed after saying this.

“Yep, not gonna argue there. I have you.”

“Ouch, that’s cute. You must do some decent good deeds to get me!”

“And I heard something like Vauju from his mouth, what’s that?”

I said, “Me his bro and you vauju, you know every one of my group calls you vauju. Literally, they never tell your name, always call you Vauju. ‘Apekshya’ equals vauju.”

“Yuck!, Vauju and me, too young to be called by it. I love my Name, call me Apekshya.”

We both laughed.

“Bibek.” She called my name like she is about to say something serious.

“Hajur, Apes,” I call her apes, so ugly in words right, she too chuckles, when I write in the message, but when Invoice, she loves being called by her nickname.

“I fear sometimes of us, you know we have this difference by, I hate this society. “

“Don’t worry, If sasura ba won’t go by our way, we do what every traditional love story couples do, elope away.”

“Be serious Nah!!” She was serious.

“I am serious k, the moment he knew that I love you, he will instantly be like, “hey Simran ji ley Apni Jindagi”. In fact, he won’t doubt to give your sister’s hand too.”

“I swear, I gonna kill you!” She laughed so hard that her elder sister too woke up.

“It’s 12, you still awake, If papa listened, you know we both die.” I was just listening.

“Didi says hi to her dearest, what you call each other, Best Friends. Why you say Best friends to everyone around me. You so ruined it’s meaning.”

I laughed, “she is my best among every others k.”. she smiled.

“I know, what you were talking this afternoon, you both circled me and my habits, I am counting each of your acts. The day, I meet you, I gonna hit you so much.”

“I miss you,” I said.

“I miss you too baby, it’s been three months since I last saw you.”

“I will be there by this New Year, baked something in my mind, you gonna love it.”

“And that is???”

“There is something called SURPRISE in the English dictionary, search its meaning tomorrow,” I said.

I feel the curiousness in her breath.

“I want Kitkats, dairymilk and so much Chocolates, with whatever you have planned. You once said, You gonna buy me chocolates every-time I smile.” And she smiled, increment the total by one.

“I remember everything, But if i could only give you one thing, in between Hug and Chocolates, what would you choose?”

“Chocolates through your lips!” I laughed.

“Okay Boss, Chocolates, lips and something special in your service. What else my Lord?”

“Just you and your love. I really miss you, I miss you so badly.”

“I know, just few months and I will be there.”

There was thundering outside, another reason I hate Pokhara, always raining cats and dogs. I sometimes think Indra Vauman has this terrible problem with Pokhreli People.

And when it thunders, I fear the most, often I sleep alongside Samir, he hates it but I am a terrible person to handle it alone, he bears this. Had to, he is my best pal.

“Is it thundering there too?” she lives in Chitwan, 140KM apart, the weather is almost the same in both the place.

“Umm, pretty badly.”

“You going to the other room to sleep?” she asked with a caring sense. It’s weird how we love others right, how hormones works when the person we care is in problem, so complex structure we develop in coarse of evolution.

“No not today may be? I want to win over my fear this time.”

“Ohh ho, Strong Boy. Be careful, if you feel uncomfortable, please don’t disturb me.”

“And now, you’re so dead.”

“I think we should sleep now, its already 1, you have to go by office tomorrow right?”

“Umm yar, I hate the Job, I wanna do something on my own, my own office, my own ideas, just few people but we build something that makes sense to the purpose of my living.”

“You will do it, I have full faith on you, One day you standing infront of thousands, inspiring them to dream high, motivating to follow the heart, I be in the corner of the room cheering for all your achievements.”

Then suddenly it hit me, this is what I said I would be during those days of colleges. But nothing has changed, same Pokhara, same rainfalls, her on the phone and the same roomie. What am I doing?

“You are my strength, Never be in the corner, always by my side at whatever I achieve.”

“I love you, talk to you tomorrow, If you say something more, I gonna cry.” She was literally on tears, the tears of love, the future we hold together that’s just grows with each sunshine.

“Bye, I love you too.”

Then as usual I went back to existential crisis…

“So, you believe in love at first sight!”. “No, not at all”, I replied. “If that’s the case, we haven’t met afterwards that day right? Then...” “Yes”. “But you just said you like me”. She asked me. “Umm, I really do like you”. I wrote with a blush in my face.

It was nine in the evening, the Chaitra of Pokhara was behaving in different way this year. Humid but more often than the previous years. Still, we unconsciously were made to comfort ourselves under a light blanket. Don’t know the reason, but looks like the global warming impact. Whatsoever, let the world concern more on it, that moment, what I know was that I was on cloud nine.

Two sad emotions forced from the other side, I could feel it. We were chatting on Viber, I pulled out the gut to ask her number last time we chatted on Facebook. At first, she didn’t give the number but at the end of the chat, she gave to me. Aside of her restrictions on calls and messages on the number, I did it. Few scolding on the initial but what more matters was her texts. I too asked her going out on “JUST FOR COFFEE” kind of date, but she always replied as a normal girl would say. “I know about guys like you!”. However she hasn’t stopped talking to me. She is different than others.

“It’s not that I started falling in love the day I first saw you, but it’s true. I like you. Your picture and you are captured in my mind since the day.’ “My picture! Ohh. Can I know how am I in the picture?” “Beautiful”. “Just beautiful?” “Precious, trust me Mona Lisa is a myth, you are the true art of god.” “You need to learn so much in praising a girl. I doubt but well tried.” “The princess has all right to doubt, but I promise to clear each doubt each time.” I thought the message this time looks so dull and incomplete, so I added few love hearts with it. We met a month ago in a street festival. She was the friend of one of my friend, I send her request, request accepted after few more days, exchanged few reactions on posts and stories. Then we started texting each other.

“From when I am your princess??”. “You are my queen”, I typed but looked so stereotype text to send. I deleted and retyped, “You are the princess of your father and mother and you were born to rule hearts of many.” This text is also not that great line, but i don’t know I strongly feel like to send the moment I typed. After sending, before the next reply, my heart pulsed so hardly. “Many?” she asked. “Prince!! “ I replied. “Thank you, mister, who trying to be prince, I guess. But these flirty lines not going to work on me! I am not like other girls.” “Thank you!” I replied. “But why????”, I could sense the urgency of answer in this reply. “For not being same as any other girls. You are just perfect the way you are. That’s the reason I fell for you.” “So finally, you admit that you fell for me. But remember, we have met just once. I may not be as good as you thinking. I may be cruel, I may be crazy, I may be mad, anything, demon beside this face.” “Oh really, I will stop following you then.” I replied to her message. “Bye! You just proved what you are.” “Sorry, sorry! Just kidding. Your par level of craziness is the pill for my life.” “I don’t believe who changes colour with time.” “I was just kidding k, oh! Princess of craziness please forgive this poor soul. I will do anything to pardon my mistake and am ready to face any punishment!” “Any punishment”. The reply was very fast. I had fallen in the trap of hers. No was not the option at all. So I texted, any punishment my lord. She laughed at my reply. “Ok! Goodnight. I am sleeping now!” “So fast, but why?” I replied within a second. “Oh! Hello, I have strict habits, I am not sparing my habits for random guys flirt.” That was her reply. “Random guy, now this hurt. Flirt is still okay.” Now, it was my turn to express my moody expression to her. “Say sorry” she replied like it was mine mistake. But why she must be angry, I must be angry as per the script, right! But no, she is angry. Girls are so unpredictable.

“Me, sorry, why, you just told me random and still you want apologies from me.” “Because you just admitted the fact that you are flirting with me.” “But with you saying me random guy, I am into tears. God going to punish you for these tears of innocent person of me.” “No, no, no! You such an idiot. Plus, dramatic and negative marking for bad acting.” “Sorry!” I apologized. She with sense of curiosity asked “Why?”. “I am not flirting; I am saying what I genuinely feel for you.” “I am also sorry for saying Random, you aren’t random guy.” She replied with few heart emoticon. “I am not random, ohh! Happiness exceeding and heart feeling something - something.” “Don’t overthink.” “Ha ha, okay. Kidding. Sleeping or changing habit for a random guy.” “Youuu!! I am sleeping, you too sleep.” “One thing!” I asked. “???” She replied. I asked either we can have meet over coffee house tomorrow at 4. “See you today if I made any mood to meet this random guy for second time.” Already past 12. “Okay, Princess, as you wish. Goodnight. See you.” “Goodnight, sweet dreams. Take care. Bye”, she texted.

Dear my first love, “I love you”, “I really do like you”. Dozens of poems and tens of stories of you and me, we just the characters of those tales, hundreds of love quotes in your name. I remember each one of those I used to keep inscribing with black ink on your name. How precious each word associated with your name, just the way you are precious to me. I hope those words were enough to describe the feelings and emotions I had in my heart for you. A shy boy with minimal social skills and possibly the outlasted & never visually seen classmate of yours’, nurturing thoughts and love in each part of his heart. Me trying to cross the boundary and break the walls, the darkest separating lines drawn between you and me. I must be foolish, I must admit it, I never deserved you, I never will, you the princess and dream girl of any guy out there and me the unsung one with zero self-esteem, zero self-confidence and literally underlying talent. You that super duper cool girl with ultimate talents and skills, and me the last among every other. Still the thoughts of being with you, to spend days where we are par from every virtual thing we owe, provoking each social beliefs that created a boundary between us, creating a world where there’s no limit of love we have for each other. You were the charm to my dull life and color to my black and white living. Yes, you were my everything, you were and is my first love. I can never explain how much I used to think of you, us being in the same square room for 2 years were the best of the days I possibly could ever have. I went unnoticed all those years but I used to think you, feel you each moment. Every fickle of seconds our eyes met intentionally or by the flavor of the time, a strong breeze of comfort stringed beats of love, the heart throbbed in the rhythm of fulfillment. Your eyes, your eyes-the perfectly curved eye-brows, the black finish of the eyeliner, crimson shyness in your face, the world within your eyes, those slender threadlike silky black hair. The art of killing with a smile aligned with pink glossy lips is something that you must have mastered in, you the professor of making fall in love for you every time I see you. And the ring-finger tip-sized red tika in your broad forehead, the bonus craft to your charm. Dear, my first love, you remember the farewell program right. The day, I, you and we separated with thousands of memories, hundreds of thoughts and dozens of sad faces. The indeed last time, I saw you, the day I literally cried of being far from you, you being far from my sight. Tears rolled down my eyes like its been never, I couldn’t show you what you meant to me, how much I adore you, how much I feel for you. A feeling of being alienated forever from your eyes started killing since then. Unfortunately, I went undiscovered that day too from your eyes. I saw you talking with literally everyone, However, me! how I went unseen? Sometimes, these sorts of memories wet my thoughts and dry my mind. The day I remember for another reason, I saw you wearing saree for the first time. The purple and sky-blue mixed saree, how someone can be beautiful as you were that day. All I have ever heard was Cinderella as a princess and angels of heaven, I bet you look gorgeous enough to make them all feel jealous. I really wish I had said “Oh! my love. You look so great today.” All-day all I been doing was tracing your presence through my eyes. Forgive me, its pervert act of mine, but the delights in your eyes made me be so. Dates and numbers freak me out, I always feel difficult to remember them. However, a date that I don’t have to force my mind to remember. Undoubtedly your date of birth, the date of March 9, 2014. The date that is something that I will never forget. The precious date of my life. Firstly, your birthday unforgettable for a reason and in another, we talked with each other for the first time. Yes, the same moment I live with each day, the same 5-second talk that keeps looping in my mind. I don’t know how I managed to heap up my guts and wish you birthday. In between others’ wishes, mine filtered but I know you replied for it. For me, more than that definite interval, the specific interval matters the most. And now, years have passed, we moved on with our pace of life, we changed and things have changed. Paths diverted to an extent from where merging us is never possible. You may or may not remember me, but I do remember you each day. You were, is and will always be my first love, your memories are not going to vanish within me till the last breath. How can I forget the girl who was the reason to attend schools, the one to heal the pain of loneliness, how can I forget the girl whose eyes whenever met with me though implicitly always made my day. You are the breeze of the spring, the color of fall, the art of the summer and beauty of the winter. In fact, you are the shadow of my living and the warmth of the chilly nights. Every drop of rain falling from the cotton sized clouds makes me remember you. Sunlight each day makes me hopeful of meeting you someday, and the nights give me dreams of you. I live you in each presence, in every second I see you. It’s hard to accept the fact you never thought me that way, you dream someone else the way I dream you, you aspire to walk holding someones else hand down the walk of life. It penetrates my heart, I sometimes cry and sadness entitles me. All I see is you and your image and thinking you to be with someone else burns my heart and soul. I almost die in every thought of yours. How painful is it to let go your beloved eloped away? I know my love is one-sided and I may never be able to confess with you but still, I can’t accept the fact of you becoming someones else queen. Lately, I have started using Instagram and started to follow you. Sorry, but stalking your profile has been part of my daily routine. Your TikTok videos and those funny posed pictures, I see all-not for once, repeatedly. I have told about you to my roommate, you know something, when I was telling him about you, I almost cried. He constantly asks me to forget you and live a life above your memories but I know my inner me will never do that though I nod my head in front of him. I see you mentioning him sometimes in your posts and uploads your couple goals memes, and sometimes those emotional sad quotes. You are still unpredictable.

It's 3 in the morning Few hours, before a bird starts to chirp A song of good morning And I am here stalking your profile Like another crazy stalker Maybe I am, but why I am? The reason you will never know.

“If I fell off tomorrow would you still love me? If I didn’t smell so good would you still hug me? If I locked up and sentenced to a quarter-century, Could I count on you to be there to support me mentally?

On Feb 12, 2020, I feel like I did something that I hadn’t done before. I gave a presentation to around 50 people with full confidence, no shivering legs, no broken mind, no sweety palm, no fumbling of words and no dry lips (Sorry, I am joking for this one). From the first slide, I felt like my lips were getting as dry as I am in the center of Sahara Desert (Shit! Bad phrase).

I was prepared with the slide a week ago, I pulled, pushed the ideas and slides from the community that I was engaged on and then merged all of it. I had that idea running in my mind, I will deliver the best presentation that everyone in the room has been seen. But, 29 freaking slides. What? Yeah, a freaking number of slides, guess how bored anyone will be in the room will be watching me trying to explain 29 slides with each point on it. The first rule of the presentation I learned at the end of the day, “Keep Slides the less as you can”. Next time, things will turn round.

The second law of presentation that I violated, “Keep memes in-between”. Fuck hell! I didn’t do it. How the hell in the world, I can miss it. Next time, a to-do list when having a presentation, Memes mandatory.

Enough jokes for the day. Let’s dive into the serious talk. My journey of being a Speaker.

Last Dashain, I applied for Microsoft Student Partner, which I wanted to be for a long time ago. I also tried for GitHub Campus expert but couldn’t be for various reasons. So, there was no sure chance of me getting an opportunity to be MSP. But last January, I got the confirmation mail from Microsoft. And I became an MSP. The biggest student partnership program in the world.

Then follow a mandatory post on Facebook. “Started job at Microsoft”. I sometimes wonder how stimulation works on our mind, (The topic let be assumed on the next blog). Congratulation received friends and family. A proud moment anyone can have as a student, to get a partnership from Microsoft. The learning path, promotion to the second stage of the program and trying hard to achieve on the third level.

Now the big day. To give a presentation for everyone who wishes to be MSP.

(That will follow on the next blog Part II of this series.)

“Ok! That’s fine, just one more warning to solve, looks like I finally finished this freaking project.”

Half-past eight and I anyhow had to show my half-asleep, half toxic and half-drunken face to my boss in an hour.

And please don’t ask me about the speed of “Pokhara Nagar Bus”. When a normal user is getting 40 Mbps of internet, the local bus makes us travel with a speed of 256 Kbps. Being a computer-guy, speed matters to me most. Every day I feel like I need to buy a vehicle myself.

Me, always in a dilemma either to seek another job in the capital or reside here congesting myself to these limited opportunities. Only that aspires me to work here is the payment, a satisfactory one, that I cannot get elsewhere for what I am. But what if I could get, no comparisons are done yet, I am too busy to where I am going.

“Holy cow! Only 30 more minutes, wasted half an hour to look like I am ready for work, which I am actually never. Just the same working function on loop, no escape back-route function declared or called.”

“Bibek! I need to talk to you. The client is not happy with your last design.”

“But…………..”

No use of saying, That was their own chosen design, The client themselves doesn’t know what they want.

“What I need to do now, then?”

“Do what they might like and no more complain after this. I want it to be final within a couple of days, we running out of dates.”

“And the project I am currently working on?”

“You have never been to the office on time and these extra excuses. I am not paying you as a senior developer for such things.”

“Oh! Yeah, I almost forgot that. keep yourself your senior tag.” I wish I said it right in his face. When you work with all the interns all around who working without payment, no pride in saying I am senior here.

How can someone think of running a company full of interns?

“Whatsoever, I will never do these kinds of unethical things, whenever I have my own start-up. Just a year more, I gonna spit on his face and move on.”

I never gonna slave someone for sake of few extra grand.

“A coffee and a cigarette.”

“And khana, shall I make it ready by now.”

“I will be here in an hour to eat.”

An infirm lifestyle in an apartment, a dimension of 4 by 4, a bed, a table, junk of wires, a chair full of to be washed clothes and a stove which I particularly don’t get time to lit up. More to it, an ashtray and bottles of hard liquors, my desert for the day.

“Hey Saurav, can you do the validation check to this?”

“I don’t know yet brother, I am learning, can you teach me how to do that?”

“Heck no! I was expecting the same, so I started doing it myself.”

Saurav, one of the interns who joined the office a week ago. His CV was full of expertise in half-dozens programming languages. No hard complaints on his CV, That’s what everyone does. “The classical Hello World output == knowledge of programming language”. He doing what everyone tries to show-off in the early days.

Inside the cubicle, the memes on Instagram and facebook look so relatable. The errors and the debugging jokes, StackOverflow and GitHub realities, and Reddit inspirations, hit me real hard.

One thing, I really care about when working on any project is privacy and security issues, I am never actually a security researcher or a pentester but I need to learn it over time.

My frustration over the work, my working ethics don’t let me release it over the normal users. The un-healthy working mechanism of the IT field in Nepal is itself a bug, it needs to be debugged first for better productivity.

“Ok! That’s fine, just one more warning to solve, looks like I finally finished this freaking project.”

Let me compile and wrap up my working directory.

bibek@BIbek:~ cd Documents/projects/deadline-December bibek@BIbek~ /Documents/projects/deadline-December g++ main.cpp -o main Segmentation fault (core dumped)

Again, same shit, the pointer issue, I absolutely hate it. One of the issues I wish never to encounter. I don’t know how many hours I am going to waste debugging this one. I am already due late to my deadlines.

“Hey! Just returned home? How was the day?”

When hours of continuous work followed by another hour of bus mess with body and mind both in a dead situation and in no mere condition to deal with further stress, these acts keep the heart kinda ‘alive’.

She was the only one who knows me and the hell I am going through. For family, I cant take a risk of quitting the job, they are proud to call their son working as a software engineer with four 0s salary.

We talk till midnight like there’s nothing more to think of. All my frustrations evaporate like a steam, a cloudy day gets the sunlight and every bit of sunlight strikes me with elegance, the sight of the moon makes me realize the world is joyful like the autumn is never coming and spring never dying.

The only part of my time when the semi-colon can be neglected in the chat lines, the only one who has no date issues, the perfect global variable, she is my number[0] with no increment function set-up.

When I myself is an alien to my own world, black terminal, and the indentation errors, where nightmares and ghosts arose in the form of bugs. All I got, it’s her, the cup of my coffee, a compiler who fixes even the run-time errors and deploys all the awesomeness to frustrations.

Oxytocin, testosterone, caffeine, nicotine, and ethanol working together in my body. A perfect combo to relax my body and soul. I have been an addict to alcohol and nicotine which I never wanted to, but the peering and payloads made me been to.

“Bibek! You again late; And you need to switch to the old design you worked on last week. The client started liking the design again and the color is also beautiful.”

While (!dead) { work(); }

The thirst in you dwells my land

My land is a slum of unprotected souls

Where your traces, protected with rubber.

//

I mastered to lure your hunger

You mastered to bargain mine

Irony, it gets a different name

You head up and I lost in day life-line.

//

Small me, hoped for an arm to rest

Every arm I now rest, sprinkle my chest

Rotten I ate were enough to live,

Fresh I now eat tears me down to hell.

//

Sometimes, a hope abides on a full moon

Light full of satisfaction asides

Canopies build stoned with affection

Which darkness pours the art, I reside.

//

The world rounds, the stars twinkle

So will the glooming moon shines,

Me sleeping with a smile underneath it

A craft that no one can take away.

//

Note: This poem is written taking a prostitute as the main character.

Overview:

Digitalization of the world has already almost reached its zenith and with evolving technologies like Artificial Intelligence, Machine Learning, Blockchain, Automated Technology, Augmented Reality, Digital Twin, 5G Network, and IoT, the template of human civilization is at its peak. If only we could put our mind into something extraordinary and then boom, technology comes into play, ideas are executed and things can happen.

Even though there`s a worldwide impact of such technologies, Nepal has still been very behind to use them for betterment. We have always relied on traditional technological tools that haven’t been productive for us. Nepal hasn’t yet invested in a prime technology which as a result has made us unable to utilize our own potential. So, it’s already time for us to take some bold moves and cultivate an environment for cutting edge technology. The Internet of Things could be the ideal one with which we can boost our way ahead.

Internet of Things:

Wikipedia defines The Internet of things (IoT) as the extension of internet connectivity into physical devices and everyday objects. IoT enables devices to interact with the internet embedded with electronics and sensors which can be controlled and monitored remotely. On a large scale, it covers other fields like AI, embedded systems and automation. It embraces traditional technologies to advanced form. This technology came to highlight a decade ago and is one of the technological trends of today.

Source: Internet

Internet of Things and Nepal:

As mentioned above, IoT could be the ideal technological investment for a country like Nepal. Progressively, things are changing. The world is taking benefit from IoT, so it’s high time to take action. Some of the ways the Nepal Government can take benefit from IoT are mentioned below;

• One of our aims is to create smart homes, smart cities, smart nation, smart cards, and ultimately smart citizens. For the very purpose, we must do some out of the box thinking and executions. The cities must be equipped with IoT controlled Traffic lights, smart parking areas, WiFi access areas, and properly planned transportation facilities. The real data manipulation and analysis will be the heart of such smart hubs. In such a scenario without the use of IoT powered technology, these concepts will only remain as words, nothing much.

• As a well-known fact, every year Nepal faces huge losses due to calamities like floods, landslides and many more. The shattering earthquake is still on our minds. Different Countries have proven to out predict them meaning its already known if the calamities will strike or not but not us, we haven’t been able to do so. It all comes down to the use of mind and tech. If we can install IoT based sensors in dams and riversides, have IoT based, sensors that can analyze the water level and speed in real-time, sensors that can calculate the speed of wind-blowing and monitor rainfall over the certain range with GIS and satellite communication could be handy in the prediction of such natural catastrophes. The following awareness about the upcoming event of disasters could save hundreds of lives and millions of wealth. Not only this, real-time communication is what IoT potent to have.

• Nepal is an agrarian country and agriculture is the sole of the income of our country. But we are facing the same problem in this sector too. We are using old-traditional tools and mechanisms in agriculture. In Agriculture, we can also Integrate IoT. If we use the Internet of Things, we can obtain real-time data of conditions like humanity, air, soil and condition fo crops. The health condition of animals can be obtained and it can be improved with the use of technology. Similarly, with the smart irrigation concept, the dependency of monsoon can be ended as technology can self-measure the requirements of crops with time resulting in high productivity with less functional costs.

• The travel and tourism industry can have a high prospect if there is a proper use of IoT based technologies. The IoT based technology can help the tourists to personalize their needs which ultimately makes the greater customer experience. Not only this they can have seamless transportation controlled by their own mobile device when the connectivity is maximized. They can travel with ease with proper location information, enjoy tourism-related sports and even get rescued in case of need.

• In addition to the above matters, IoT will create an environment for new investment in the country that will unlock our potential to interact with third countries of the world.

Conclusion:

The data collected in real-time will be beneficial and act as a data set for future references. The data so collected when used in ML and AI-based algorithms will make a significant result. Such results could be helpful in the planning of developmental projects. IoT is future technology and the Nepal government must invest in it for the sake of competing with other countries. Long in the future, wars and competitions will be backed by cyber-weapons, and IoT is one of them for developing countries like Nepal, which can serve on high note only if we invested some time and manpower in it.

How someone could feel so much of close just in fickle of seconds” She did that.

Source: Internet

Slight drizzle followed by the sudden cats and dogs making the way for a downpour. unusual in a way, November doesn’t call cloudburst. But this time, it did with a sense of humor in the air.

Pokhara, badly known for untimely rainfall added one more incident to its hall of fame. I don’t call it an honor, I better call it disgust. Basically, I hate rain, it disturbs isolation which I prefer the most, I believe in Punctuality. Rainfall sucks the hell out of the word “Punctuality”, I am not that stereotypes who says, it natural. But Nature needs to be punctual, right? Nevermind, I don’t need your answer. All I need is that coffee I ordered a few minutes ago.

I am possibly in the best coffeehouse in Pokhara. If I am addicted to something, it’s always been caffeine and fighting blank paper to ink love and lust thoughts. I remember once someone asked me why I am so dedicated to these niches. I replied, ‘“Everyone fantasizes what they haven’t got yet”. Enough inner-me talks. Let’s return to the moment. Finally here comes my coffee Mocha, all thanks to the punctual waitress.

Grabbing my umbrella and my airpods I started walking home. A blue kurta girl soaked in rain, trying not to further. Was she playing hide and seek with the rain? Aah, nostalgic. My legs paralyzed with that sight of her. An oval face with celestial nose in between those upturned eyes perfectly set upon the base of heart faced lips. On top of that, her faux waterfall hairstyle of which few passing by the middle of right eyes even after she collects and accumulates with rest. Really an adorable face to remember under the rainfall when minimal light is lighting up the place. Indeed I saw a charming face to craft a poem on.

“Oh! I am………”

“Hello Are your listening”

/A deep silence/

A wave, in fact, a strong wave to make sure I am still there. She lends me a hand to shake, I did as she says.

“And yours?”

“?”

“Name?”

“? Name”

“Is there a name or something that you are called by?”

“Oh! Sorry, Bibek.”

Had she told her name? Shit, shit I missed it.

Even in the rain, the heat is melting me down. My heart’s racing and feet shivering. This has never happened before.

/Silence for a minute/

“Bibek, Bibek”, a sudden striking pat on my shoulder. Back to reality. I did lose my past, where she told her name. If I could travel that alternate dimension and relive that past, nevertheless it is just another word to sympathize with the reality.

“You okay with me sharing your umbrella, looks like you are dead lost”. I wasn’t I conformed.

“So you too love rain.”

I remained silent.

“I understood, you don’t want to talk to a stranger like me”

“No, it’s not like that.”

“Just kidding,’’ she said.

Yes, she isn’t a stranger, she is at present the most known girl of my life. I myself am a stranger, it feels like I know her for centuries. How someone could feel so much of close just in fickle of seconds” She did that, I am inflated by her. She made me lost within the moment.

“So you love rain as I do.”

“No!” /I shouldn’t have gone with a single word./

“Why?”

“Just don’t like.”

“Oh! Boy.”

She grabbed my hand, took the shaft of my umbrella and took away from me. All of sudden, those freaking cubic centimeters water drops engulfed me. The rain made a way to the ground through my body. I wet, the same unpunctual rainfall poured its existence upon me.

But more to this, I didn’t feel bad about it because she was there enjoying it. The rain was happy to finally found its counterpart in hide and seek. Now, will the rain die and she will act as a seeker or its just a one-way game. She always hides, just the rain seeks her to furnish its power within her. Whatsoever is this game, she is enjoying.

“Do you know, I have this strange allergy to rainfall.

My body can’t endure rainwater for long, sulfate ions are dreadful for my skin. Yet, I like to soak up in the rain just because my soul loves to explore the world within each raindrop. It’s funny how my body and soul diverge. I listen to my heart. Such a weirdo me, right?”

/She continued./

“I love the rain for various reasons. It molds us life learning lessons, It verse us to be pure, to adapt to the changes around us, to let go and live life to the fullest. In fact, it makes us realize whatever we think we are, we will all end up beneath the ground. Why suffer and show stingy nature when the end is sure.”

“Did you enjoy wetting in the mizzle.”

I nodded.

“Oh! Silencer boy, don’t your heart says to talk. You are something different.”

I didn’t realize I had no time. If only I had known, maybe I`d speak some words and make a conversation with her. But soon she said, “Here’s my place; now I have to go. Thanks for the moment Bibek. You are surely a person to remember. See you. Bye”

After she was gone, some part of me hated myself. I should’ve talked, she seemed as if i knew her forever. Thoughts started popping in mind. “Are we going to meet again? When, how, why? “What you seek is seeking you”, but I`m not even sure if I do seek her. I’ve always somehow hated rain, so is the way of nature telling me guilty?

I couldn’t help but think this is me. I always ruin good things. Hell, how can I not like these rainfalls? I always pretend to be aloof, a dysfunctional society survivor and maybe I`m. Although even just for a moment, I feel like I lost myself there.

“Above the traces of crying clouds

Two souls hooding the same canopy

Piercing boundaries of strangers

Exploding themselves in each drop of rain………”

After century of first world war, a new war for supremacy has started between two countries again. China and America are on a cold war to gain financial as well as technological domination around the world. And the next big weapon for that is ‘5G’ technology.

According to 3GPP(The third generation partnership project), which is associated to maintain standard protocols to mobile technology and works for maintenance and development of GSM and related 2G, 2.5G etc, 5G technology is defined as any system using 5GNR(5G new radio). The speed of 5G will be 100 times to that 4G, which will lower the lag-time by 5 times, increases mobile data by 1000 times and enable new capabilities to unlock innovation. In simple language, it’s less the concern of download speed and more of a game changer.

The first fairly substantial deployments were in April 2019. SK Telecoms, KT corporations and LG U PLUS, Telecommunications operators based on South Korea were among the first to install base station. Verizon, also claimed to provide services in two of the major cities of USA. But that was initial testing, the final deployment hasn’t been made yet. China and America are on the track to be the first to provide 5G service around their respective nations. Whoever wins the race is likely to dominate both technological and financial market.

With 4G technology only, America increased their GDP by $100 billions in 2016 which created more jobs, lowered costs of consumer goods and helped US to roughly earned around $125 billions in revenue. And 5G will have even bigger impact to this sector around the globe. It is estimated that 5G promises to make benefit of around $12 trillion and America expect to make benefit of $3.5 trillion alone. So being first will eventually make change in economic sector.

The technology advancement from 4G to 5G is just not limited to number, it’s impact is way beyond that. There are many technological projects around the world whose effective functionality is not possible by the band-width and speed of 4G technology can finally be out of prototype. VIrtual Reality will finally be a reality, wireless technology will substantially replace the cable technology, AI based technologies will be empowered effectively, gaming experience will be even better, Health care will have improvement with better tracking of patience health condition and much more, Internet Of Things will be used with ease, “smart city, smart home, smart citizens and smart living” concepts will be propelled. More or Less, The world will be same as the movies based on future years to come.

China has invested $17.7 billion in capital so far and added more than 350,000 wireless facilities since 2015, giving the country about 1.9 million wireless sites. Not only this China has planed to invest roughly around $500 billion in 5G-related investment over coming five years. Similarly, America is about to invest $100 billion till 2020.

For months, Australia had warned the United States about the destructive capacity of 5G technology. Now, America is aggressively campaigning against Chinese telecom champion Huawei, fearful Beijing’s domination of 5G could be used for espionage and sabotage.* *Huawei, giant technological corporation based on China has been banned by US following allegation of theft of intellectual property, obstruction of justice and fraud related to its alleged evasion of US sanctions against Iran. That will surely slow the race for the China at least for few times. When US banned Huawei, Chinese rejected Apple product which divided the giants symmetrically. The gap of mutual understanding has made the race even beautiful for third nations.

“In this 5G war, we can tell that no matter for a country or a company, everyone is striving to develop 5G capacities internally while keeping an eye on the external competition.” An statement written in Official Huawei consumer page.

According to the report conducted by CTIA, the US leads with 92 commercial 5G deployments planned for 2019 but noted that China’s wireless operators are conducting “hundreds of large-scale 5G trials across the country.” But the China has has infrastructural advantage over the US as China has more than 14 cell sites per 10000 people compared to 4.7 of US.According to the survey, South Korea ranked third in 5G readiness, followed by Japan, Britain and Italy. But there is a saying in English “pioneers take the arrows, settlers take the land”, that means whoever is leading now, one that settles for the long will eventually win the race.

Refrences:

1. Wikipedia (www.wikipedia.com)

2. Google (www.google.com)

3. Forbes (www.forbes.com)

4. Reuters (www.reuters.com) etc.