{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Previous Page]()                                                [Next Page]()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise: Some More Algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook shows another class comparison between cuML and Scikit-learn: The `LogisticRegression`. The basic form of logistic regression is used to model the probability of a certain class or event happening based on a set of variables.\n",
    "\n",
    "We also use this as an example of how cuML can adapt to other GPU centric workflows, this time based on CuPy, a GPU centric NumPy like library for array manipulation: [CuPy](https://cupy.chainer.org)\n",
    "\n",
    "Thanks to the [CUDA Array Interface](https://numba.pydata.org/numba-doc/dev/cuda/cuda_array_interface.html) cuML is compatible with multiple GPU memory libraries that conform to the spec, and tehrefore can use objects from libraries such as CuPy or Pytorch without additional memory copies!\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Here is the list of exercises and modules in the lab:\n",
    "<a href='#lr'>Logistic Regression</a><br>\n",
    "- <a href='#ex1'>Exercise 1</a><br>\n",
    "- <a href='#ex2'>Exercise 2</a><br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='lr'></a>\n",
    "Lets begin by importing our needed libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "# Lets use cupy in a similar fashion to how we use numpy\n",
    "import cupy as cp\n",
    "\n",
    "from sklearn import metrics, datasets\n",
    "from sklearn.linear_model import LogisticRegression as skLogistic\n",
    "from sklearn.preprocessing import binarize\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.colors import ListedColormap\n",
    "\n",
    "cm_bright = ListedColormap(['#FF0000', '#0000FF'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, lets use Scikit-learn to create a dataset to use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = datasets.make_classification(10000, n_features=2, n_informative=2, n_redundant=0, \n",
    "                                     n_clusters_per_class=1, class_sep=0.5, random_state=1485)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets create our `X` and `y` arrays in CuPy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = cp.array(a[0], order='F') # the API of CuPy is almost identical to NumPy\n",
    "y = cp.array(a[1], order='F')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets see how the dataset works:    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 42324 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(cp.asnumpy(X[:,0]), cp.asnumpy(X[:,1]), c=[cm_bright.colors[i] for i in cp.asnumpy(y)], \n",
    "            alpha=0.1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets divide our dataset into training and testing datasets in a simple manner:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the data into a training and test set using NumPy like syntax\n",
    "X_train = X[:8000, :].copy(order='F')\n",
    "X_test = X[-2000:, :].copy(order='F')\n",
    "y_train = y[:8000]\n",
    "y_test = y[8000:10000]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the resulting objects are still CuPy arrays in GPU: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "cupy.core.core.ndarray"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.__class__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise: Fit the cuML and Scikit-learn `LogisticRegression` objects and compare them when they use as similar parameters as possible\n",
    "\n",
    "* Hint 1: the **default values** of parameters in cuML are **the same** as the default values for Scikit-learn most of the time, so we recommend to leave all parameters except for `solver` as the default \n",
    "\n",
    "\n",
    "* Hint 2: Remember the **solver can differ significantly between the libraries**, so look into the solvers offered by both libraries to make them match \n",
    "\n",
    "\n",
    "* Hint 3: Even though Scikit-learn expects Numpy objects, it **cannot** accept CuPy objects for many of its methods since it expects the memory to be on CPU (host), not on GPU (device)\n",
    "\n",
    "For convenience, the notebook offers a few cells to organize your work."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='ex1'></a>\n",
    "\n",
    "### 1. Fit Scikit-learn LogisticRegression and show its accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary><b>Solution for Scikit-learn</b></summary>\n",
    "   <pre>\n",
    "\n",
    "clf = skLogistic()\n",
    "clf.fit(X_train.get(), y_train.get())\n",
    "clf.score(X_test.get(), y_test.get())\n",
    "</pre>\n",
    "</details>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='ex2'></a>\n",
    "\n",
    "### 2. Fit cuML Regression and show its accuracy\n",
    "\n",
    "* Hint 1: Look at the data types expected by cuML methods: https://rapidsai.github.io/projects/cuml/en/stable/api.html#cuml.LogisticRegression.fit \n",
    "   one or more of the input vectors might not be of the expected data type! You may need to typecast.\n",
    "\n",
    "\n",
    "\n",
    "* Hint 2: as mentioned above, cuML has native support for CuPy objects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details><summary><b>Solution for CuML</b></summary>\n",
    "   <pre>\n",
    "\n",
    "reg = LogisticRegression()\n",
    "reg.fit(X_train,y_train)\n",
    "\n",
    "print(\"Coefficients:\")\n",
    "print(reg.coef_)\n",
    "print(\"Intercept:\")\n",
    "print(reg.intercept_)\n",
    "\n",
    "preds = reg.predict(X_test)\n",
    "\n",
    "print(preds)\n",
    "\n",
    "print('Scikit-learn accuracy: ' + str(reg.score(X_test, y_test)))\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cuml import LogisticRegression\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coefficients:\n",
      "[[-1.28726705]\n",
      " [ 2.75051631]]\n",
      "Intercept:\n",
      "[-0.812162]\n",
      "Scikit-learn accuracy: 0.8025000095367432\n"
     ]
    }
   ],
   "source": [
    "# useful methods: cupy_array.astype(np_dtype) converts an array from one datatype to np_datatype, where np_datatype can be something like np.float32, np.float64, etc.\n",
    "# useful methods: cudf_seris.to_array() converts a cuDF Series to a numpy array\n",
    "# useful methods: cp.asnumpy(cupy_array) converts cupy to numpy\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Expected accuracies for apples to apples comparison: 0.8025 vs 0.8695**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additional Exercise: Play with the different parameters, particularly the different Scikit-learn solvers to see how they differ in behavior even in the same library!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Licensing\n",
    "  \n",
    "This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[[1]]()\n",
    "[[2]]()\n",
    "[[3]]()\n",
    "[[4]]()\n",
    "[[5]]()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
