{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Previous Notebook](Part_2.ipynb)\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "[Home Page](../Start_Here.ipynb)\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Deep Learning - Residual Networks \n",
    "\n",
    "In this notebook, participants will be introduced to Residual Networks, and it's terminologies.\n",
    "\n",
    "**Contents of the this notebook:**\n",
    "\n",
    "- [Residual Networks](#Residual-Networks)\n",
    "- [Residual Blocks](##Residual-Blocks)\n",
    "- [Building Resnet](#Building-our-Resnet)\n",
    "\n",
    "\n",
    "**By the end of this notebook participants will:**\n",
    "\n",
    "- Understand the need for Residual Networks\n",
    "- Understand Resnets  \n",
    "- Write own Resnet Classifier and train it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Residual Networks\n",
    "\n",
    "In CNN's, we learned that the learning is hierarchical, each increase in the number of layers results in more complex features being learned by the layers. But despite this, it is shown empirically that there is a maximum threshold for depth with the traditional CNN model.\n",
    "\n",
    "In a paper titled [Deep Residual Learning for Image Recognition](https://arxiv.org/pdf/1512.03385.pdf) researchers from Microsoft pointed out the following :\n",
    "<img src=\"images/resnet.PNG\">\n",
    "\n",
    "The failure of the 56-layer CNN could be blamed on the optimization function, initialization of the network, or the famous vanishing/exploding gradient problem. Vanishing gradients are exceptionally easy to blame for this.\n",
    "\n",
    "But what is **Vanishing Gradient**? Now when we do Back-propagation, the gradients tend to get smaller and smaller as we keep on moving backwards in the network. This means that the neurons in the earlier layers learn very slowly as compared to the neurons in the later layers in the hierarchy. The earlier layers in the network are slowest to train.\n",
    "\n",
    "Earlier layers in the network are essential because they are responsible for learning and detecting the simple patterns and are actually the building blocks of our network. Obviously, if they give improper and inaccurate results, then how can we expect the next layers and the complete network to perform nicely and produce accurate results.\n",
    "\n",
    "\n",
    "The problem of training very deep networks has been alleviated with the introduction of a new neural network layer — The Residual Block.\n",
    "\n",
    "**Optional - The Degradation Problem:** \n",
    "\n",
    "The degradation problem suggests that the solvers might have difficulties in approximating identity mappings by multiple nonlinear layers with the residual learning reformulation. \n",
    "\n",
    "Let us consider network A having $n$ layers and Network B having $m$ layers given that $  n > m $ , if Network A perform poorly relative to Network B, one might argue Network A being larger than Network B, if it could map an Identity Function for the First $ ( n-m ) $ layers, then it would have performed in par with Network B. Still, it doesn't learn it due to the Vanishing Gradients problems, so here when we use Residual Networks, the network gets the Input along with the learning on the Residual and if the Input Function was the Appropriate Function, it could quickly change the weights of the Residual Function to be Zero.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Residual Block \n",
    "\n",
    "In residual block the activation of a layer is fast-forwarded to a deeper layer in the neural network. Residual Blocks help in the flow of information from the initial layers to the final layers. This is done by the introduction of skip connections, as seen in the image below.\n",
    "\n",
    "*Let us consider $H(x)$  as an underlying mapping to be fit by a few stacked layers (not necessarily the entire net), with $x$ denoting the inputs to the first of these layers. If one hypothesizes that multiple nonlinear layers can asymptotically approximate complicated functions, then it is equivalent to hypothesize that they can asymptotically approximate the residual functions* \n",
    "\n",
    "i.e., $H(x) − x$ (assuming that the input and output are of the same dimensions). \n",
    "\n",
    "So rather than expecting the stacked layers to approximate $H(x)$, we explicitly let these layers approximate a residual function. \n",
    "\n",
    "$F(x) = H(x) − x$. \n",
    "\n",
    "The original function thus becomes $F(x)+x$\n",
    "\n",
    "*Although both models should be able to approximate the desired functions asymptotically, the ease of learning might be different. This reformulation is motivated by the counterintuitive phenomena about the degradation problem. As we discussed above, if the added layers can be constructed as identity mappings, a deeper model should have training error no greater than its shallow counterpart. *\n",
    "\n",
    "<img src=\"images/resblock.PNG\">\n",
    "\n",
    "With this, the increase in the number of layers adds to the improvements in accuracy. Here are some results from the paper:\n",
    "\n",
    "<img src=\"images/stats.png\">\n",
    "\n",
    "Now let us see how to write a Residual Block in Keras."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building our Residual Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Image Classification on types of clothes  \n",
    "\n",
    "####  Step -1 : Data \n",
    "\n",
    "We will be using the **F-MNIST ( Fashion MNIST )** dataset. This dataset contains 70,000 grayscale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels).\n",
    "\n",
    "<img src=\"images/fashion-mnist.png\" alt=\"Fashion MNIST sprite\"  width=\"600\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import Necessary Libraries\n",
    "\n",
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "\n",
    "# TensorFlow and tf.keras\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras.layers import Input, Add, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D, AveragePooling2D, MaxPooling2D, GlobalMaxPooling2D\n",
    "from tensorflow.keras.models import Model, load_model\n",
    "from tensorflow.keras.preprocessing import image\n",
    "from tensorflow.keras.applications.imagenet_utils import preprocess_input\n",
    "from IPython.display import SVG\n",
    "from tensorflow.keras.utils import plot_model\n",
    "from tensorflow.keras.initializers import glorot_uniform\n",
    "import scipy.misc\n",
    "from matplotlib.pyplot import imshow\n",
    "%matplotlib inline\n",
    "\n",
    "# Helper libraries\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's Import the Dataset\n",
    "fashion_mnist = keras.datasets.fashion_mnist\n",
    "\n",
    "(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Defining Class Names\n",
    "class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
    "               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Printing the Size of our Dataset\n",
    "\n",
    "#Print Array Size of Training Set \n",
    "print(\"Size of Training Images :\"+str(train_images.shape))\n",
    "#Print Array Size of Label\n",
    "print(\"Size of Training Labels :\"+str(train_labels.shape))\n",
    "\n",
    "#Print Array Size of Test Set \n",
    "print(\"Size of Test Images :\"+str(test_images.shape))\n",
    "#Print Array Size of Label\n",
    "print(\"Size of Test Labels :\"+str(test_labels.shape))\n",
    "\n",
    "#Let's See how our Outputs Look like \n",
    "print(\"Training Set Labels :\"+str(train_labels))\n",
    "#Data in the Test Set\n",
    "print(\"Test Set Labels :\"+str(test_labels))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data Pre-processing \n",
    "\n",
    "plt.figure()\n",
    "plt.imshow(train_images[0])\n",
    "plt.colorbar()\n",
    "plt.grid(False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_images = train_images / 255.0\n",
    "test_images = test_images / 255.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's Print to Veryify if the Data is of the correct format.\n",
    "plt.figure(figsize=(10,10))\n",
    "for i in range(25):\n",
    "    plt.subplot(5,5,i+1)\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "    plt.grid(False)\n",
    "    plt.imshow(train_images[i], cmap=plt.cm.binary)\n",
    "    plt.xlabel(class_names[train_labels[i]])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reshape input data from (28, 28) to (28, 28, 1)\n",
    "w, h = 28, 28\n",
    "train_images = train_images.reshape(train_images.shape[0], w, h, 1)\n",
    "test_images = test_images.reshape(test_images.shape[0], w, h, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model : \n",
    "\n",
    "Let's build a Resnet Model with Keras and train it.\n",
    "\n",
    "We will be using two kinds of Residual Blocks :\n",
    "    - Identity Block \n",
    "    - Convolution Block\n",
    "    \n",
    "#### Identity Block :     \n",
    "\n",
    "In the *Identity Block* we have a skip connection with no change in input. \n",
    "\n",
    "<img src=\"images/identity.png\">\n",
    "\n",
    "\n",
    "#### Convolution Block : \n",
    "\n",
    "Convolutional block is almost same as identity block but there is a convolutional layer in short-cut path to just change the dimension such that the dimension of input and output matches.\n",
    "\n",
    "<img src=\"images/conv.png\">\n",
    "\n",
    "\n",
    "Let's Start building the Identity Block :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def identity_block(X, f, filters, stage, block):\n",
    "    \n",
    "    # Defining name basis\n",
    "    conv_name_base = 'res' + str(stage) + block + '_branch'\n",
    "    bn_name_base = 'bn' + str(stage) + block + '_branch'\n",
    "    \n",
    "    # Retrieve Filters\n",
    "    F1, F2, F3 = filters\n",
    "    \n",
    "    # A Path is a Block of Conv followed by Batchnormalization and Actication\n",
    "    # Save the input value\n",
    "    X_shortcut = X\n",
    "    \n",
    "    # First component of main path\n",
    "    X = Conv2D(filters = F1, kernel_size = (1, 1), strides = (1,1), padding = 'valid', name = conv_name_base + '2a', kernel_initializer = glorot_uniform(seed=0))(X)\n",
    "    X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X)\n",
    "    X = Activation('relu')(X)\n",
    "    \n",
    "    # Second component of main path\n",
    "    X = Conv2D(filters = F2, kernel_size = (f, f), strides = (1, 1), padding = 'same', name = conv_name_base + '2b', kernel_initializer = glorot_uniform(seed=0))(X)\n",
    "    X = BatchNormalization(axis = 3, name = bn_name_base + '2b')(X)\n",
    "    X = Activation('relu')(X)\n",
    "\n",
    "    # Third component of main path \n",
    "    X = Conv2D(filters = F3, kernel_size = (1, 1), strides = (1, 1), padding = 'valid', name = conv_name_base + '2c', kernel_initializer = glorot_uniform(seed=0))(X)\n",
    "    X = BatchNormalization(axis = 3, name = bn_name_base + '2c')(X)\n",
    "\n",
    "    # Final step: Add shortcut value to main path, and pass it through a RELU activation\n",
    "    X = Add()([X, X_shortcut])\n",
    "    X = Activation('relu')(X)\n",
    "    \n",
    "    return X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convolution Block \n",
    "\n",
    "Notice the only change we need to do is adding a convolution and batch normalisation for the input data to match the output dimension. \n",
    "\n",
    "This can be done by adding the following lines : \n",
    "\n",
    "```python\n",
    "##### SHORTCUT PATH #### \n",
    "    X_shortcut = Conv2D(filters=F3, kernel_size=(1, 1), strides=(s, s), padding='valid', name=conv_name_base + '1', kernel_initializer=glorot_uniform(seed=0))(X_shortcut)\n",
    "    X_shortcut = BatchNormalization(axis=3, name=bn_name_base + '1')(X_shortcut)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convolutional_block(X, f, filters, stage, block, s=2):\n",
    "\n",
    "    # Defining name basis\n",
    "    conv_name_base = 'res' + str(stage) + block + '_branch'\n",
    "    bn_name_base = 'bn' + str(stage) + block + '_branch'\n",
    "\n",
    "    # Retrieve Filters\n",
    "    F1, F2, F3 = filters\n",
    "\n",
    "    # Save the input value\n",
    "    X_shortcut = X\n",
    "\n",
    "    ##### MAIN PATH #####\n",
    "    # First component of main path \n",
    "    X = Conv2D(filters=F1, kernel_size=(1, 1), strides=(s, s), padding='valid', name=conv_name_base + '2a', kernel_initializer=glorot_uniform(seed=0))(X)\n",
    "    X = BatchNormalization(axis=3, name=bn_name_base + '2a')(X)\n",
    "    X = Activation('relu')(X)\n",
    "\n",
    "    # Second component of main path\n",
    "    X = Conv2D(filters=F2, kernel_size=(f, f), strides=(1, 1), padding='same', name=conv_name_base + '2b', kernel_initializer=glorot_uniform(seed=0))(X)\n",
    "    X = BatchNormalization(axis=3, name=bn_name_base + '2b')(X)\n",
    "    X = Activation('relu')(X)\n",
    "\n",
    "    # Third component of main path\n",
    "    X = Conv2D(filters=F3, kernel_size=(1, 1), strides=(1, 1), padding='valid', name=conv_name_base + '2c', kernel_initializer=glorot_uniform(seed=0))(X)\n",
    "    X = BatchNormalization(axis=3, name=bn_name_base + '2c')(X)\n",
    "\n",
    "    ##### SHORTCUT PATH #### \n",
    "    X_shortcut = Conv2D(filters=F3, kernel_size=(1, 1), strides=(s, s), padding='valid', name=conv_name_base + '1', kernel_initializer=glorot_uniform(seed=0))(X_shortcut)\n",
    "    X_shortcut = BatchNormalization(axis=3, name=bn_name_base + '1')(X_shortcut)\n",
    "\n",
    "    # Final step: Add shortcut value to main path, and pass it through a RELU activation\n",
    "    X = Add()([X, X_shortcut])\n",
    "    X = Activation('relu')(X)\n",
    "\n",
    "    return X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ResNet(input_shape = (28, 28, 1), classes = 10):\n",
    "    \n",
    "    # Define the input as a tensor with shape input_shape\n",
    "    X_input = Input(shape=input_shape)\n",
    "\n",
    "    \n",
    "    # Zero-Padding\n",
    "    X = ZeroPadding2D((3, 3))(X_input)\n",
    "    \n",
    "    # Stage 1\n",
    "    X = Conv2D(64, (7, 7), strides = (2, 2), name = 'conv1', kernel_initializer = glorot_uniform(seed=0))(X)\n",
    "    X = BatchNormalization(axis = 3, name = 'bn_conv1')(X)\n",
    "    X = Activation('relu')(X)\n",
    "    X = MaxPooling2D((3, 3), strides=(2, 2))(X)\n",
    "\n",
    "    # Stage 2\n",
    "    X = convolutional_block(X, f = 3, filters = [64, 64, 256], stage = 2, block='a', s = 1)\n",
    "    X = identity_block(X, 3, [64, 64, 256], stage=2, block='b')\n",
    "    X = identity_block(X, 3, [64, 64, 256], stage=2, block='c')\n",
    "\n",
    "    # Stage 3\n",
    "    X = convolutional_block(X, f=3, filters=[128, 128, 512], stage=3, block='a', s=2)\n",
    "    X = identity_block(X, 3, [128, 128, 512], stage=3, block='b')\n",
    "    X = identity_block(X, 3, [128, 128, 512], stage=3, block='c')\n",
    "    X = identity_block(X, 3, [128, 128, 512], stage=3, block='d')\n",
    "\n",
    "    # AVGPOOL\n",
    "    X = AveragePooling2D(pool_size=(2,2), padding='same')(X)\n",
    "\n",
    "    # Output layer\n",
    "    X = Flatten()(X)\n",
    "    X = Dense(classes, activation='softmax', name='fc' + str(classes), kernel_initializer = glorot_uniform(seed=0))(X)\n",
    "    \n",
    "    \n",
    "    # Create model\n",
    "    model = Model(inputs = X_input, outputs = X, name='ResNet')\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras import backend as K\n",
    "K.clear_session()\n",
    "model = ResNet(input_shape = (28, 28, 1), classes = 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.fit(train_images, train_labels, epochs = 5, batch_size = 32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Evaluating the Model using the Test Set\n",
    "\n",
    "test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=2)\n",
    "\n",
    "print('\\nTest accuracy:', test_acc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Predictions : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Making Predictions from the test_images\n",
    "\n",
    "predictions = model.predict(test_images)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reshape input data from (28, 28) to (28, 28, 1)\n",
    "w, h = 28, 28\n",
    "train_images = train_images.reshape(train_images.shape[0], w, h)\n",
    "test_images = test_images.reshape(test_images.shape[0], w, h)\n",
    "\n",
    "# Helper Functions to Plot Images \n",
    "def plot_image(i, predictions_array, true_label, img):\n",
    "  predictions_array,true_label, img = predictions_array, true_label[i], img[i]\n",
    "  plt.grid(False)\n",
    "  plt.xticks([])\n",
    "  plt.yticks([])\n",
    "\n",
    "  plt.imshow(img, cmap=plt.cm.binary)\n",
    "\n",
    "  predicted_label = np.argmax(predictions_array)\n",
    "  if predicted_label == true_label:\n",
    "    color = 'blue'\n",
    "  else:\n",
    "    color = 'red'\n",
    "\n",
    "  plt.xlabel(\"{} {:2.0f}% ({})\".format(class_names[predicted_label],\n",
    "                                100*np.max(predictions_array),\n",
    "                                class_names[true_label]),\n",
    "                                color=color)\n",
    "\n",
    "def plot_value_array(i, predictions_array, true_label):\n",
    "  predictions_array, true_label = predictions_array, true_label[i]\n",
    "  plt.grid(False)\n",
    "  plt.xticks(range(10))\n",
    "  plt.yticks([])\n",
    "  thisplot = plt.bar(range(10), predictions_array, color=\"#777777\")\n",
    "  plt.ylim([0, 1])\n",
    "  predicted_label = np.argmax(predictions_array)\n",
    "\n",
    "  thisplot[predicted_label].set_color('red')\n",
    "  thisplot[true_label].set_color('blue')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the first X test images, their predicted labels, and the true labels.\n",
    "# Color correct predictions in blue and incorrect predictions in red.\n",
    "num_rows = 5\n",
    "num_cols = 3\n",
    "num_images = num_rows*num_cols\n",
    "plt.figure(figsize=(2*2*num_cols, 2*num_rows))\n",
    "for i in range(num_images):\n",
    "  plt.subplot(num_rows, 2*num_cols, 2*i+1)\n",
    "  plot_image(i, predictions[i], test_labels, test_images)\n",
    "  plt.subplot(num_rows, 2*num_cols, 2*i+2)\n",
    "  plot_value_array(i, predictions[i], test_labels)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion : \n",
    "\n",
    "Running both our models for 5 Epochs here is a table comparing them : \n",
    "\n",
    "|  Model   | Train Accuracy  | Train Loss  | Test Accuracy | Test Loss |\n",
    "|----------|-----------------|-------------|---------------|-----------|\n",
    "| Fully Connected Network - After 5 Epochs  |            0.8923 |       0.2935 |       0.8731  |     0.2432|           \n",
    "| Convolutional Network - After 5 Epochs  |              0.8860|       0.3094 |         0.9048 |       0.1954 | \n",
    "| Residual Network - After 5 Epochs | 0.9064| 0.2610 | 0.8713 | 0.3398|\n",
    "\n",
    "* Note : These Values can be different when you run it.\n",
    "\n",
    "So, from the table above we can conclude that for this example how CNNs are efficient compared to other Machine Learning algorithms when it comes to image processing tasks.\n",
    "\n",
    "Congrats on coming this far, now that you are introduced to Machine Learning and Deep Learning, you can get started on the domain specific problem accessible through the Home Page.\n",
    "\n",
    "## Important:\n",
    "<mark>Shutdown the kernel before clicking on “Next Notebook” to free up the GPU memory.</mark>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reference \n",
    "\n",
    "[The Vanishing Gradient Problem](https://medium.com/@anishsingh20/the-vanishing-gradient-problem-48ae7f501257)\n",
    "\n",
    "[Introduction to ResNets](https://towardsdatascience.com/introduction-to-resnets-c0a830a288a4)\n",
    "\n",
    "[Hitchhiker’s Guide to Residual Networks (ResNet) in Keras](https://towardsdatascience.com/hitchhikers-guide-to-residual-networks-resnet-in-keras-385ec01ec8ff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Previous Notebook](CNN's.ipynb)\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "[Home Page](../Start_Here.ipynb)\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;&emsp;&emsp;\n",
    "&emsp;&emsp;&emsp;"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
