{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[Home Page](../Start_Here.ipynb)\n", "     \n", "     \n", "     \n", "     \n", "   \n", "[Next Notebook](CNN's.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# CNN Primer and Keras 101" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "In this notebook you will be introduced to the concept of a convolutional neural network (CNN) and implement one using Keras. This notebook is designed as a starting point for absolute beginners to deep learning.\n", "\n", "**Contents of this notebook:**\n", "\n", "- [How a deep learning project is planned](#Machine-Learning-Pipeline)\n", "- [Wrapping things up with an example (classification)](#Wrapping-Things-up-with-an-Example)\n", " - [Fully connected networks](#Image-Classification-on-types-of-Clothes)\n", "\n", "\n", "**By the end of this notebook the participant will:**\n", "\n", "- Understand machine learning pipelines\n", "- Write a deep learning classifier and train it\n", "\n", "**We will be building a _multi-class classifier_ to classify images of clothing into their respective classes.**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Machine Learning Pipeline\n", "\n", "During the bootcamp we will be making use of the following concepts to help us understand how a machine learning (ML) project should be planned and executed:\n", "\n", "1. **Data**: To start any ML project we need data which is pre-processed and can be fed into the network.\n", "2. **Task**: There are many possible tasks in the field of ML; we need to make sure we understand and define the problem statement accurately.\n", "3. **Model**: We need to build our model, which is neither too deep (requiring a lot of computational power) nor too small (preventing it from learning the important features).\n", "4. **Loss**: Out of the many _loss functions_ that can be defined, we need to carefully choose one which is suitable for the task we are about to carry out.\n", "5. **Learning**: There are a variety of _optimisers_, each with their advantages and disadvantages. We must choose one which is suitable for our task and train our model using some suitably chosen hyperparameters.\n", "6. **Evaluation**: We must determine if our model has learned the features properly by analysing how it performs on data it has not previously seen. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Here we will be building a _multi-class classifier_ to classify images of clothing into their respective classes.**\n", "\n", "We will follow the pipeline presented above to complete the example.\n", "\n", "## Image classification on types of clothes \n", "\n", "#### Step 1: Data \n", "\n", "We will be using the **Fashion MNIST** dataset, which is a very popular introductory dataset in deep learning. This dataset contains 70,000 grayscale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels).\n", "\n", "\"Fashion\n", "\n", "*Source: https://www.tensorflow.org/tutorials/keras/classification*" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Import Necessary Libraries\n", "\n", "from __future__ import absolute_import, division, print_function, unicode_literals\n", "\n", "# TensorFlow and tf.keras\n", "import tensorflow as tf\n", "from tensorflow import keras\n", "\n", "# Helper libraries\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "print(tf.__version__)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# Let's Import the Dataset\n", "fashion_mnist = keras.datasets.fashion_mnist\n", "\n", "(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Loading the dataset returns four NumPy arrays:\n", "\n", "* The `train_images` and `train_labels` arrays are the *training set*—the data the model uses to learn.\n", "* The model is tested against the *test set*, the `test_images`, and `test_labels` arrays.\n", "\n", "The images are 28x28 NumPy arrays, with pixel values ranging from 0 to 255. The *labels* are an array of integers, ranging from 0 to 9. These correspond to the *class* of clothing the image represents:\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
LabelClass
0T-shirt/top
1Trouser
2Pullover
3Dress
4Coat
5Sandal
6Shirt
7Sneaker
8Bag
9Ankle boot
\n", "\n", "Each image is mapped to a single label. Since the *class names* are not included with the dataset, let us store them in an array so that we can use them later when plotting the images:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n", " 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Understanding the Data" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Print array size of training dataset\n", "print(\"Size of Training Images: \" + str(train_images.shape))\n", "# Print array size of labels\n", "print(\"Size of Training Labels: \" + str(train_labels.shape))\n", "\n", "# Print array size of test dataset\n", "print(\"Size of Test Images: \" + str(test_images.shape))\n", "# Print array size of labels\n", "print(\"Size of Test Labels: \" + str(test_labels.shape))\n", "\n", "# Let's see how our outputs look\n", "print(\"Training Set Labels: \" + str(train_labels))\n", "# Data in the test dataset\n", "print(\"Test Set Labels: \" + str(test_labels))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data Preprocessing\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "plt.figure()\n", "plt.imshow(train_images[0])\n", "plt.colorbar()\n", "plt.grid(False)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The image pixel values range from 0 to 255. Let us now normalise the data range from 0 - 255 to 0 - 1 in both the *Train* and *Test* set. This normalisation of pixels helps us by optimizing the process where the gradients are computed." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "train_images = train_images / 255.0\n", "test_images = test_images / 255.0" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Let's print to verify whether the data is of the correct format.\n", "plt.figure(figsize=(10,10))\n", "for i in range(25):\n", " plt.subplot(5,5,i+1)\n", " plt.xticks([])\n", " plt.yticks([])\n", " plt.grid(False)\n", " plt.imshow(train_images[i], cmap=plt.cm.binary)\n", " plt.xlabel(class_names[train_labels[i]])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Defining our Model\n", "\n", "Our model has three layers:\n", "\n", "- 784 input features (28 * 28)\n", "- 128 nodes in the hidden layer (feel free to experiment with this value)\n", "- 10 output nodes to denote the class\n", "\n", "We will implement this model in Keras (TensorFlow's high-level API for machine learning).\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from tensorflow.keras import backend as K\n", "K.clear_session()\n", "model = keras.Sequential([\n", " keras.layers.Flatten(input_shape=(28, 28)),\n", " keras.layers.Dense(128, activation='relu'),\n", " keras.layers.Dense(10, activation='softmax')\n", "])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first layer in this network, `tf.keras.layers.Flatten`, transforms the format of the images from a two-dimensional array (of 28 by 28 pixels) to a one-dimensional array (of 28 * 28 = 784 pixels). Think of this layer as unstacking rows of pixels in the image and lining them up. This layer has no parameters to learn; it only reformats the data.\n", "\n", "After the pixels are flattened, the network consists of a sequence of two `tf.keras.layers.Dense` layers. These are densely connected, or fully connected, neural layers. The first `Dense` layer has 128 nodes (or neurons). The second (and last) layer is a 10-node *softmax* layer that returns an array of 10 probability scores that sum to 1. Each node contains a score that indicates the probability that the current image belongs to one of the 10 classes.\n", "\n", "### Compile the model\n", "\n", "Before the model is ready for training, it needs a few more settings. These are added during the model's *compile* step:\n", "\n", "* *Loss function* —This measures how accurate the model is during training. You want to minimize this function to \"steer\" the model in the right direction.\n", "* *Optimizer* —This is how the model is updated based on the data it sees and its loss function.\n", "* *Metrics* —Used to monitor the training and testing steps. The following example uses *accuracy*, the fraction of the images that are correctly classified." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model.compile(optimizer='adam',\n", " loss='sparse_categorical_crossentropy',\n", " metrics=['accuracy'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train the model\n", "\n", "Training the neural network model requires the following steps:\n", "\n", "1. Feed the training data to the model. In this example, the training data is in the `train_images` and `train_labels` arrays.\n", "2. The model learns to associate images and labels.\n", "3. You ask the model to make predictions about a test set—in this example, the `test_images` array. Verify that the predictions match the labels from the `test_labels` array.\n", "\n", "To start training, call the `model.fit` method—so called because it \"fits\" the model to the training data:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model.fit(train_images, train_labels, epochs=5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Evaluate accuracy\n", "\n", "Next, compare how the model performs on the test dataset:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Evaluating the model using the test dataset\n", "\n", "test_loss, test_acc = model.evaluate(test_images, test_labels, verbose=2)\n", "\n", "print('\\nTest accuracy:', test_acc)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We get an accuracy of 87% on the test dataset, which is less than the 89% we got during the training phase. This problem in machine learning is called overfitting.\n", "\n", "## Exercise\n", "\n", "Try adding more dense layers to the network above and observe the change in accuracy.\n", "\n", "## Important:\n", "Shut down the kernel before clicking on “Next Notebook” to free up the GPU memory.\n", "\n", "\n", "## Licensing\n", "This material is released by OpenACC-Standard.org, in collaboration with NVIDIA Corporation, under the Creative Commons Attribution 4.0 International (CC BY 4.0)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[Home Page](../Start_Here.ipynb)\n", "     \n", "     \n", "     \n", "     \n", "   \n", "[Next Notebook](CNN's.ipynb)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.2" } }, "nbformat": 4, "nbformat_minor": 2 }