{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[Home Page](../Start_Here.ipynb)\n", "     \n", "     \n", "     \n", "     \n", "   \n", "[Next Notebook](CNN's.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# CNN Primer and Keras 101" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "In this notebook, participants will be introduced to CNN, implement it using Keras. For an absolute beginner this notebook would serve as a good starting point.\n", "\n", "**Contents of the this notebook:**\n", "\n", "- [How a Deep Learning project is planned ?](#Machine-Learning-Pipeline)\n", "- [Wrapping things up with an example ( Classification )](#Image-Classification-on-types-of-clothes)\n", "\n", "\n", "**By the end of this notebook participant will:**\n", "\n", "- Understand the Machine Learning Pipeline\n", "- Write a Deep Learning Classifier and train it.\n", "\n", "**We will be building a _Multi-class Classifier_ to classify images of clothing to their respective classes**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Machine Learning Pipeline\n", "\n", "During the bootcamp we will be making use of the following buckets to help us understand how a Machine Learning project should be planned and executed: \n", "\n", "1. **Data**: To start with any ML project we need data which is pre-processed and can be fed into the network.\n", "2. **Task**: There are many tasks present in ML, we need to make sure we understand and define the problem statement accurately.\n", "3. **Model**: We need to build our model, which is neither too deep and complex, thereby taking a lot of computational power or too small that it could not learn the important features.\n", "4. **Loss**: Out of the many _loss functions_ present, we need to carefully choose a _loss function_ which is suitable for the task we are about to carry out.\n", "5. **Learning**: As we mentioned in our last notebook, there are a variety of _optimisers_ each with their advantages and disadvantages. So here we choose an _optimiser_ which is suitable for our task and train our model using the set hyperparameters.\n", "6. **Evaluation**: This is a crucial step in the process to determine if our model has learnt the features properly by analysing how it performs when unseen data is given to it. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Here we will be building a _Multi-class Classifier_ to classify images of clothing to their respective classes.**\n", "\n", "We will follow the above discussed pipeline to complete the example.\n", "\n", "## Image Classification on types of clothes \n", "\n", "#### Step -1 : Data \n", "\n", "We will be using the **F-MNIST ( Fashion MNIST )** dataset, which is a very popular dataset. This dataset contains 70,000 grayscale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels).\n", "\n", "\"Fashion\n", "\n", "*Source: https://www.tensorflow.org/tutorials/keras/classification*" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Import Necessary Libraries\n", "\n", "from __future__ import absolute_import, division, print_function, unicode_literals\n", "\n", "# TensorFlow and tf.keras\n", "import tensorflow as tf\n", "from tensorflow import keras\n", "\n", "# Helper libraries\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "print(tf.__version__)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Let's Import the Dataset\n", "fashion_mnist = keras.datasets.fashion_mnist\n", "\n", "(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Loading the dataset returns four NumPy arrays:\n", "\n", "* The `train_images` and `train_labels` arrays are the *training set*—the data the model uses to learn.\n", "* The model is tested against the *test set*, the `test_images`, and `test_labels` arrays.\n", "\n", "The images are 28x28 NumPy arrays, with pixel values ranging from 0 to 255. The *labels* are an array of integers, ranging from 0 to 9. These correspond to the *class* of clothing the image represents:\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
LabelClass
0T-shirt/top
1Trouser
2Pullover
3Dress
4Coat
5Sandal
6Shirt
7Sneaker
8Bag
9Ankle boot
\n", "\n", "Each image is mapped to a single label. Since the *class names* are not included with the dataset, let us store them in an array so that we can use them later when plotting the images:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n", " 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Understanding the Data" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Print Array Size of Training Set \n", "print(\"Size of Training Images :\"+str(train_images.shape))\n", "#Print Array Size of Label\n", "print(\"Size of Training Labels :\"+str(train_labels.shape))\n", "\n", "#Print Array Size of Test Set \n", "print(\"Size of Test Images :\"+str(test_images.shape))\n", "#Print Array Size of Label\n", "print(\"Size of Test Labels :\"+str(test_labels.shape))\n", "\n", "#Let's See how our Outputs Look like \n", "print(\"Training Set Labels :\"+str(train_labels))\n", "#Data in the Test Set\n", "print(\"Test Set Labels :\"+str(test_labels))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data Pre-processing\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "plt.figure()\n", "plt.imshow(train_images[0])\n", "plt.colorbar()\n", "plt.grid(False)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The image pixel values range from 0 to 255. Let us now normalise the data range from 0 - 255 to 0 - 1 in both the *Train* and *Test* set. This Normalisation of pixels helps us by optimizing the process where the gradients are computed." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "train_images = train_images / 255.0\n", "test_images = test_images / 255.0" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Let's Print to Veryify if the Data is of the correct format.\n", "plt.figure(figsize=(10,10))\n", "for i in range(25):\n", " plt.subplot(5,5,i+1)\n", " plt.xticks([])\n", " plt.yticks([])\n", " plt.grid(False)\n", " plt.imshow(train_images[i], cmap=plt.cm.binary)\n", " plt.xlabel(class_names[train_labels[i]])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Defining our Model\n", "\n", "Our Model has three layers :\n", "\n", "- 784 Input features ( 28 * 28 ) \n", "- 128 nodes in hidden layer (Feel free to experiment with the value)\n", "- 10 output nodes to denote the Class\n", "\n", "Implementing the same in Keras ( Machine Learning framework built on top of Tensorflow, Theano, etc..) \n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras import backend as K\n", "K.clear_session()\n", "model = keras.Sequential([\n", " keras.layers.Flatten(input_shape=(28, 28)),\n", " keras.layers.Dense(128, activation='relu'),\n", " keras.layers.Dense(10, activation='softmax')\n", "])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first layer in this network, `tf.keras.layers.Flatten`, transforms the format of the images from a two-dimensional array (of 28 by 28 pixels) to a one-dimensional array (of 28 * 28 = 784 pixels). Think of this layer as unstacking rows of pixels in the image and lining them up. This layer has no parameters to learn; it only reformats the data.\n", "\n", "After the pixels are flattened, the network consists of a sequence of two `tf.keras.layers.Dense` layers. These are densely connected, or fully connected, neural layers. The first `Dense` layer has 128 nodes (or neurons). The second (and last) layer is a 10-node *softmax* layer that returns an array of 10 probability scores that sum to 1. Each node contains a score that indicates the probability that the current image belongs to one of the 10 classes.\n", "\n", "### Compile the model\n", "\n", "Before the model is ready for training, it needs a few more settings. These are added during the model's *compile* step:\n", "\n", "* *Loss function* —This measures how accurate the model is during training. You want to minimize this function to \"steer\" the model in the right direction.\n", "* *Optimizer* —This is how the model is updated based on the data it sees and its loss function.\n", "* *Metrics* —Used to monitor the training and testing steps. The following example uses *accuracy*, the fraction of the images that are correctly classified." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model.compile(optimizer='adam',\n", " loss='sparse_categorical_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train the model\n", "\n", "Training the neural network model requires the following steps:\n", "\n", "1. Feed the training data to the model. In this example, the training data is in the `train_images` and `train_labels` arrays.\n", "2. The model learns to associate images and labels.\n", "3. You ask the model to make predictions about a test set—in this example, the `test_images` array. Verify that the predictions match the labels from the `test_labels` array.\n", "\n", "To start training, call the `model.fit` method—so called because it \"fits\" the model to the training data:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model.fit(train_images, train_labels ,epochs=5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate accuracy\n", "\n", "Next, compare how the model performs on the test dataset:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Evaluating the Model using the Test Set\n", "\n", "test_loss, test_acc = model.evaluate(test_images, test_labels, verbose=2)\n", "\n", "print('\\nTest accuracy:', test_acc)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise\n", "\n", "Try adding more dense layers to the network above and observe change in accuracy." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We get an Accuracy of 87% in the Test dataset which is less than the 89% we got during the Training phase, This problem in ML is called as Overfitting\n", "\n", "## Important:\n", "Shutdown the kernel before clicking on “Next Notebook” to free up the GPU memory.\n", "\n", "## Licensing\n", "This material is released by OpenACC-Standard.org, in collaboration with NVIDIA Corporation, under the Creative Commons Attribution 4.0 International (CC BY 4.0)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[Home Page](../Start_Here.ipynb)\n", "     \n", "     \n", "     \n", "     \n", "   \n", "[Next Notebook](CNN's.ipynb)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.2" } }, "nbformat": 4, "nbformat_minor": 2 }