{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "     \n", "     \n", "     \n", "     \n", "     \n", "   \n", "[Home Page](../START_HERE.ipynb)\n", "\n", "[Previous Notebook](01-LinearRegression-Hyperparam.ipynb)\n", "     \n", "     \n", "     \n", "     \n", "[1](01-LinearRegression-Hyperparam.ipynb)\n", "[2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# CuML Exercise\n", "Scikit-Learn is an incredibly powerful toolkit that allows data scientists to quickly build models from their data, and it one of the most common and useful tools in the Python data science ecosystem. cuML is the RAPIDS library that implements similar machine learning algorithms that use CUDA to run on GPUs, with an API that mirrors the Scikit-learn one as much as possible.\n", "\n", "In this notebook we present a small exercise for new users to experiment with CuML and apply their knowledge on a real world machine learning dataset. We will be working on the Car Accidents dataset that we started preprocessing in the CuDF tutorial. This is a countrywide car accident dataset, which covers 49 states of the USA. The accident data are collected from February 2016 to June 2020, using two APIs that provide streaming traffic incident (or event) data. These APIs broadcast traffic data captured by a variety of entities, such as the US and state departments of transportation, law enforcement agencies, traffic cameras, and traffic sensors within the road-networks. Currently, there are about 3.5 million accident records in this dataset. If you skipped that tutorial, you can download the processed dataset here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Challenge\n", "\n", "We begin by perfoming some data manipulation using Scikit learn preprocessing and removing any class imbalance. The actual exercise begins here, where we have provided the implementation of 4 different Scikit-learn models and you have to convert them to CuML and evaluate the performance difference." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first step is downloading the dataset and putting it in the data directory, for using in this tutorial. Download the dataset here, and place it in (host/data) folder. Now we will import the necessary libraries." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np; print('NumPy Version:', np.__version__)\n", "%matplotlib inline\n", "import sys\n", "import sklearn; print('Scikit-Learn Version:', sklearn.__version__)\n", "from sklearn.linear_model import LinearRegression\n", "\n", "from sklearn import preprocessing \n", "import pandas as pd\n", "from sklearn.utils import resample\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.model_selection import GridSearchCV\n", "from sklearn.feature_selection import SelectFromModel\n", "from sklearn.metrics import classification_report, confusion_matrix, accuracy_score, roc_curve, auc\n", "from sklearn.preprocessing import OrdinalEncoder, StandardScaler\n", "import cudf\n", "import cupy\n", "\n", "# import for visualization\n", "import matplotlib.pyplot as plt\n", "\n", "# import for model building\n", "from sklearn.svm import SVC\n", "from sklearn.tree import DecisionTreeClassifier\n", "from sklearn.ensemble import RandomForestClassifier\n", "from sklearn.neighbors import KNeighborsClassifier\n", "from cuml.linear_model import MBSGDRegressor as cumlSGD\n", "from sklearn.linear_model import SGDRegressor as skSGD\n", "from sklearn.datasets import make_regression\n", "from sklearn.metrics import mean_squared_error\n", "\n", "from cuml.ensemble import RandomForestClassifier as curfc\n", "from sklearn.ensemble import RandomForestClassifier as skrfc\n", "\n", "from cuml import make_regression\n", "from cuml.linear_model import LinearRegression as cuLinearRegression\n", "from cuml.metrics.regression import r2_score\n", "from sklearn.linear_model import LinearRegression as skLinearRegression\n", "\n", "from cuml.neighbors import KNeighborsClassifier as KNeighborsC\n", "from sklearn.neighbors import KNeighborsClassifier\n", "from cuml.linear_model import MBSGDClassifier as cumlMBSGDClassifier\n", "from sklearn.linear_model import SGDClassifier\n", "from sklearn.preprocessing import StandardScaler\n", "from sklearn.pipeline import make_pipeline\n", "from cuml import Ridge\n", "from cuml.linear_model import Ridge\n", "from sklearn.linear_model import Ridge\n", "from cuml import LogisticRegression\n", "from sklearn.linear_model import LogisticRegression as skLogistic\n", "from cuml.linear_model import ElasticNet\n", "from sklearn import linear_model\n", "\n", "from cuml.linear_model import Lasso\n", "from cuml.solvers import SGD as cumlSGD" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's read the dataframe from the csv which was processed in the previous tutorial and stored in the data folder." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%time df = pd.read_csv('../../data/data_proc.csv')\n", "print(df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Drop the unnecessary columns which got added while reading the file." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "df = df.drop(columns = [\"Unnamed: 0\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Observe the dataset by printing the first 5 rows using the head function." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Drop any null values that may be present." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "df = df.dropna()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We are continuing a bit of the preprocessing that is easier using Scikit-learn and can use Label encoding to convert the labels to numbers without increasing the dimensions of our dataset. Label encoder converts the string categorical values to numbers. Eg. [Chicago, New York, Mumbai] would get encoded to [0, 1, 2]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "#link to label encoder\n", "label_encoder = preprocessing.LabelEncoder() \n", "df['County']= label_encoder.fit_transform(df['County']) \n", "df['State']= label_encoder.fit_transform(df['State'])\n", "df['Weather_Condition']= label_encoder.fit_transform(df['Weather_Condition'])\n", "\n", "df['Source'] = label_encoder.fit_transform(df['Source'])\n", "\n", "df['Sunrise_Sunset'] = label_encoder.fit_transform(df['Sunrise_Sunset'])\n", "df['Civil_Twilight'] = label_encoder.fit_transform(df['Civil_Twilight'])\n", "df['Nautical_Twilight'] = label_encoder.fit_transform(df['Nautical_Twilight'])\n", "df['Astronomical_Twilight'] = label_encoder.fit_transform(df['Astronomical_Twilight'])\n", "\n", "df['Amenity'] = label_encoder.fit_transform(df['Amenity'])\n", "df['Bump'] =label_encoder.fit_transform(df['Bump'])\n", "df['Crossing'] = label_encoder.fit_transform(df['Crossing'])\n", "df['Give_Way'] = label_encoder.fit_transform(df['Give_Way'])\n", "df['Junction'] =label_encoder.fit_transform(df['Junction'])\n", "df['No_Exit'] = label_encoder.fit_transform(df['No_Exit'])\n", "df['Railway'] = label_encoder.fit_transform(df['Railway'])\n", "df['Roundabout'] = label_encoder.fit_transform(df['Roundabout'])\n", "\n", "df['Station'] = label_encoder.fit_transform(df['Station'])\n", "df['Stop'] = label_encoder.fit_transform(df['Stop'])\n", "df['Traffic_Calming'] = label_encoder.fit_transform(df['Traffic_Calming'])\n", "df['Traffic_Signal'] = label_encoder.fit_transform(df['Traffic_Signal'])\n", "df['Turning_Loop'] =label_encoder.fit_transform(df['Turning_Loop'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's continue with exploring the dataset. We can check how the values are distributed in different categories." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "df['Severity'].value_counts()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The distribution across all the severities is imbalanced and Machine Learning algorithms tend to produce unsatisfactory classifiers when faced with imbalanced datasets.So we will convert this dataset to the necessary form by performing class balancing using up sampling. Up-sampling is the process of randomly duplicating observations from the minority class in order to reinforce its signal.\n", "\n", "- First, we'll separate observations from each class into different DataFrames.\n", "- Next, we'll resample the minority class with replacement, setting the number of samples to match that of the majority class.\n", "- Finally, we'll combine the up-sampled minority class DataFrame with the original majority class DataFrame." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "# Class Balancing | Using Up Sampling\n", "\n", "# Separate majority and minority classes\n", "df_s1 = df[df['Severity']==1]\n", "df_s2 = df[df['Severity']==2]\n", "df_s3 = df[df['Severity']==3]\n", "df_s4 = df[df['Severity']==4]\n", "\n", "count = max(df_s1.count()[0], df_s2.count()[0], df_s3.count()[0], df_s4.count()[0])\n", "\n", "# Upsample minority class\n", "df_s1 = resample(df_s1, replace=df_s1.count()[0]\n", "\n", "#### Your exercise begins here. Provided below are 4 ML models in Scikit-learn, which you have to convert to CuML and evaluate the performance difference.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Logistic Regression\n", "\n", "Logistic regression is a statistical model that in its basic form uses a logistic function to model a binary dependent variable.\n", "\n", "## Scikit-learn\n", "\n", "### Fit" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "clf = skLogistic()\n", "clf.fit(X_train, y_train)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "print(clf.score(X_test, y_test))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Implement the code above in CuML
\n", "\n", "## CuML\n", "\n", "### Fit" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Modify the code in this cell\n", "\n", "%%time\n", "reg = LogisticRegression()\n", "reg.fit() # Pass the train cudf dataframes as arguments here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Modify the code in this cell\n", "\n", "%%time\n", "print(reg.score()) # Pass the test cudf dataframes as arguments here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Nearest Neighbours Classifier\n", "\n", "NearestNeighbors implements unsupervised nearest neighbors learning. It acts as a uniform interface to three different nearest neighbors algorithms: BallTree, KDTree, and a brute-force algorithm based on routines in sklearn.metrics.pairwise. The choice of neighbors search algorithm is controlled through the keyword 'algorithm', which must be one of ['auto', 'ball_tree', 'kd_tree', 'brute']. When the default value 'auto' is passed, the algorithm attempts to determine the best approach from the training data.\n", "\n", "## Scikit-learn\n", "\n", "### Fit" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "neigh = KNeighborsClassifier(n_neighbors=3)\n", "neigh.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "print(neigh.score(X_test, y_test))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Implement the code above in CuML
\n", "\n", "## CuML\n", "\n", "### Fit" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Modify the code in this cell\n", "\n", "%%time\n", "knn = KNeighborsC(n_neighbors=10)\n", "knn.fit() # Pass the train cudf dataframes as arguments here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Modify the code in this cell\n", "\n", "%%time\n", "print(knn.score()) # Pass the test cudf dataframes as arguments here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## ElasticNet Classifier\n", "\n", "Elastic Net first emerged as a result of critique on lasso, whose variable selection can be too dependent on data and thus unstable. The solution is to combine the penalties of ridge regression and lasso to get the best of both worlds. Ridge Regression, which penalizes sum of squared coefficients (L2 penalty). Lasso Regression, which penalizes the sum of absolute values of the coefficients (L1 penalty).\n", "\n", "### Scikit-learn model\n", "\n", "#### Fit" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "regr = ElasticNet()\n", "regr.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Evaluate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "X_test = X_test.astype(np.float64)\n", "y_test = y_test.astype(np.float64)\n", "print(regr.score(X_test,y_test))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " Implement the code above in CuML
\n", "\n", "### CuML model\n", "\n", "#### Fit" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Modify the code in this cell\n", "\n", "%%time\n", "enet = ElasticNet()\n", "\n", "enet.fit() # Pass the train cudf dataframes as arguments here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Modify the code in this cell\n", "\n", "%%time\n", "X_cudf_test = X_cudf_test.astype(np.float64)\n", "y_cudf_test = y_cudf_test.astype(np.float64)\n", "print(enet.score()) # Pass the test cudf dataframes as arguments here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# CONCLUSION" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's compare the performance of our solution! Write down your observations after converting the code to CuML and compare with Scikit-learn implementation." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#Modify the code in this cell below" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "| Algorithm | Implementation | Accuracy | Time | Algorithm | Implementation | Accuracy | Time |\n", "| ----------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Wow! This was an interesting exercise. We hope you enjoyed applying your machine learning skills and appreciated the GPU boost provided by RAPIDS. CuML supports many ML models which can provide interesting results on this dataset. If you faced difficulties, refer to the sample solution notebook [here](04_CuML_Solution.ipynb)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# References\n", "\n", "- Moosavi, Sobhan, Mohammad Hossein Samavatian, Srinivasan Parthasarathy, and Rajiv Ramnath. “A Countrywide Traffic Accident Dataset.”, 2019.\n", "\n", "- Moosavi, Sobhan, Mohammad Hossein Samavatian, Srinivasan Parthasarathy, Radu Teodorescu, and Rajiv Ramnath. \"Accident Risk Prediction based on Heterogeneous Sparse Data: New Dataset and Insights.\" In proceedings of the 27th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM, 2019.\n", "\n", "- If you need to refer to the dataset, you can download it [here](https://www.kaggle.com/sobhanmoosavi/us-accidents)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\"Creative

This dataset is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Licensing\n", " \n", "This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "     \n", "     \n", "     \n", "     \n", "     \n", "   \n", "[Home Page](../START_HERE.ipynb)\n", "\n", "[Previous Notebook](01-LinearRegression-Hyperparam.ipynb)\n", "     \n", "     \n", "     \n", "     \n", "[1](01-LinearRegression-Hyperparam.ipynb)\n", "[2]" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.2" } }, "nbformat": 4, "nbformat_minor": 4 }