{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "<p align=\"center\">\n",
    "    <img src=\"https://github.com/GeostatsGuy/GeostatsPy/blob/master/TCG_color_logo.png?raw=true\" width=\"220\" height=\"240\" />\n",
    "</p>\n",
    "\n",
    "## Interactive Linear Regression in Python for Engineers, Data Scientists and Geoscientists \n",
    "\n",
    "### Michael Pyrcz, Professor, The University of Texas at Austin \n",
    "\n",
    "#### Contacts: [Twitter/@GeostatsGuy](https://twitter.com/geostatsguy) | [GitHub/GeostatsGuy](https://github.com/GeostatsGuy) | [www.michaelpyrcz.com](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446)\n",
    "\n",
    "This is an interactive dashboard to demonstrate the training (fitting the model to training data) for **Linear Regression** in Python with the SciPy package, [Stats](https://docs.scipy.org/doc/scipy/reference/stats.html) module.    \n",
    "\n",
    "##### My Other Resources\n",
    "\n",
    "I have not included all the details, e.g., for the test assumptions in this document. These are the related lectures with linked Python interactive dashboards and well-documented demonstrations (like this one):\n",
    "\n",
    "* [Bivariate Statistics](https://youtu.be/wZwYEDqB4A4)\n",
    "* [Q-Q plots](https://youtu.be/RETZus4XBNM) \n",
    "* [Linear Regression](https://youtu.be/HzWq7pj2F_w) \n",
    "* [Norms](https://youtu.be/JmxGlrurQp0)\n",
    "\n",
    "#### Load the required libraries\n",
    "\n",
    "The following code loads the required libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os                                                   # to set current working directory \n",
    "import numpy as np                                          # arrays and matrix math\n",
    "import scipy.stats as st                                    # statistical methods\n",
    "from sklearn.linear_model import LinearRegression           # scikit-learning multilinear regression\n",
    "import pandas as pd                                         # DataFrames\n",
    "import matplotlib.pyplot as plt                             # for plotting\n",
    "plt.rc('axes', axisbelow=True)                              # set axes and grids in the background for all plots\n",
    "import math                                                 # for square root"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.  \n",
    "\n",
    "#### Declare Functions\n",
    "\n",
    "For the more complicated operations (e.g., prediction intervals), I made functions for readable code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the working directory\n",
    "\n",
    "I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).  Also, in this case make sure to place the required (see below) data file in this working directory.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#os.chdir(\"C:\\PGE337\")                                      # set the working directory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using You Own Data\n",
    "\n",
    "I have set up this workflow so you can easily replace my provided demonstration dataset with your own data.\n",
    "\n",
    "1. Replace the file to load below with your own file. You could load from the cloud or a local file, I have provided examples for both.\n",
    "\n",
    "2. Below, specify the column from your DataFrame below for the predictor feature, X, and the response feature, y.\n",
    "\n",
    "3. Also, set the range of the predictor and response features for plotting,and the features' name and features' units.\n",
    "\n",
    "#### Loading Data\n",
    "\n",
    "Let's load the provided dataset. 'Density_Por_data.csv' is available at https://github.com/GeostatsGuy/GeoDataSets. It is a comma delimited file with 105 porosity and density measures from a rock unit.\n",
    "\n",
    "* density ($g/cm^3$)\n",
    "* porosity ($\\%$). \n",
    "\n",
    "We load it with the pandas 'read_csv' function into a data frame we called 'df' and then preview it by printing a slice and by utilizing the 'head' DataFrame member function (with a nice and clean format, see below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Working with 10 data.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Density</th>\n",
       "      <th>Porosity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>2.216014</td>\n",
       "      <td>9.376752</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>2.090105</td>\n",
       "      <td>10.743471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>1.768993</td>\n",
       "      <td>11.531035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1.734373</td>\n",
       "      <td>13.981934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>1.552713</td>\n",
       "      <td>14.120720</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Density   Porosity\n",
       "33  2.216014   9.376752\n",
       "76  2.090105  10.743471\n",
       "75  1.768993  11.531035\n",
       "9   1.734373  13.981934\n",
       "40  1.552713  14.120720"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#df = pd.read_csv(\"Density_Por_data.csv\")                    # read a .csv file in as a DataFrame\n",
    "df = pd.read_csv(r\"https://raw.githubusercontent.com/GeostatsGuy/GeoDataSets/master/Density_Por_data.csv\") # load data from Dr. Pyrcz's GitHub repository\n",
    "df = df.sample(frac = 0.1,replace = False, random_state = 17)                   # random sample without replacement, for fewer data and easier visualization\n",
    "print('Working with ' + str(len(df)) + ' data.')\n",
    "df.head()                                                    # we could also use this command for a table preview "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is useful to review the summary statistics of our loaded DataFrame.  That can be accomplished with the 'describe' DataFrame member function.  We transpose to switch the axes for ease of visualization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Density</th>\n",
       "      <td>10.0</td>\n",
       "      <td>1.834286</td>\n",
       "      <td>0.433337</td>\n",
       "      <td>1.06796</td>\n",
       "      <td>1.598128</td>\n",
       "      <td>1.854829</td>\n",
       "      <td>2.184537</td>\n",
       "      <td>2.339324</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Porosity</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.561390</td>\n",
       "      <td>3.525219</td>\n",
       "      <td>4.96824</td>\n",
       "      <td>9.453360</td>\n",
       "      <td>11.137253</td>\n",
       "      <td>14.086023</td>\n",
       "      <td>16.786859</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          count       mean       std      min       25%        50%        75%  \\\n",
       "Density    10.0   1.834286  0.433337  1.06796  1.598128   1.854829   2.184537   \n",
       "Porosity   10.0  11.561390  3.525219  4.96824  9.453360  11.137253  14.086023   \n",
       "\n",
       "                max  \n",
       "Density    2.339324  \n",
       "Porosity  16.786859  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe().transpose()                                    # calculate summary statistics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we extract the samples from the DataFrame into separate 1d ndarrays called 'por' and 'den' for convenience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predictor feature: Density, label: Density ($g/cm^3$)\n",
      "Response feature: Porosity, label: Porosity (%)\n"
     ]
    }
   ],
   "source": [
    "y = df['Porosity'].values                                  # shallow copy features to 1D ndarrays for convenience \n",
    "yname = 'Porosity'; yunits = '%'\n",
    "ylabel = 'Porosity ('+ yunits +')'\n",
    "ymin = 0.0; ymax = 25.0\n",
    "X = df['Density'].values; xunits = '$g/cm^3$'\n",
    "xname = 'Density'; \n",
    "xlabel = 'Density (' + xunits + ')'\n",
    "xmin = 1.0; xmax = 2.4\n",
    "dX = np.linspace(xmin,xmax,100)                              # a 1D array of 100 density values for calculation and plotting\n",
    "\n",
    "print('Predictor feature: ' + xname + ', label: ' + (xlabel))\n",
    "print('Response feature: ' + yname + ', label: ' + ylabel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Linear Regression Model\n",
    "\n",
    "Let's first calculate the linear regression model, a model object from SciPy package, stats module.\n",
    "\n",
    "This is our model:\n",
    "\n",
    "\\begin{equation}\n",
    "por = b_1 \\times den + b_0\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Output from linregress function is <class 'scipy.stats._stats_mstats_common.LinregressResult'>:\n",
      "LinregressResult(slope=-7.382413926717521, intercept=25.102846211945106, rvalue=-0.9074828304159511, pvalue=0.00028629737143429134, stderr=1.2082490694086176, intercept_stderr=2.271253484351185)\n",
      "\n",
      "Linear Regression Model:\n",
      "  Slope: -7.38\n",
      "  Intercept: 25.1\n",
      "  Square Root of R-squared: -0.91\n",
      "  Slope p-value: 0.0\n",
      "  Slope standard error: 1.21\n"
     ]
    }
   ],
   "source": [
    "linear = st.linregress(X,y)                              # output is the linear regression model\n",
    "print('\\nOutput from linregress function is ' + str(type(linear)) + ':')\n",
    "print(linear)\n",
    "print('\\nLinear Regression Model:')\n",
    "print('  Slope: ' + str(round(linear.slope,2)))\n",
    "print('  Intercept: ' + str(round(linear.intercept,2)))\n",
    "print('  Square Root of R-squared: ' + str(round(linear.rvalue,2)))\n",
    "print('  Slope p-value: ' + str(round(linear.pvalue,2)))\n",
    "print('  Slope standard error: ' + str(round(linear.stderr,2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 96916 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "aslope = 6.0\n",
    "\n",
    "\n",
    "yhat = linear.intercept + linear.slope*X\n",
    "ayhat = linear.intercept + aslope*X\n",
    "dif_mean = np.average(y) - np.average(ayhat)\n",
    "aint = linear.intercept + dif_mean\n",
    "ayhat2 = aint + aslope*X\n",
    "\n",
    "plt.subplot(131)\n",
    "plt.scatter(X, y,color='black',alpha=0.8,facecolor='grey',label='sample data',zorder=10)\n",
    "plt.scatter(X, yhat,color='black',s=40,facecolor='darkorange',alpha=0.8,label='model',zorder=10)\n",
    "plt.plot(dX, linear.intercept + linear.slope*dX, 'black', label='linear regression model',zorder=1)\n",
    "\n",
    "for i in range(0,len(X)):\n",
    "    plt.plot([X[i],X[i]],[y[i],yhat[i]],c='grey',alpha=0.8,ls='--',zorder=1)\n",
    "plt.title('Sample Data and Least Squares Best Fit Linear Regression Model'); plt.xlabel(xlabel); plt.ylabel(ylabel)\n",
    "plt.legend()\n",
    "plt.grid(True); plt.xlim([1.0,2.4]); plt.ylim([5,20])\n",
    "\n",
    "plt.subplot(132)\n",
    "plt.scatter(X, y,color='black',alpha=0.8,facecolor='grey',label='sample data',zorder=10)\n",
    "plt.scatter(X, ayhat2,color='black',s=40,facecolor='darkorange',alpha=0.8,label='our model',zorder=10)\n",
    "plt.plot(dX, aint + aslope*dX, 'black', label='our linear model',zorder=1)\n",
    "\n",
    "for i in range(0,len(X)):\n",
    "    plt.plot([X[i],X[i]],[y[i],ayhat2[i]],c='grey',alpha=0.8,ls='--',zorder=1)\n",
    "plt.title('Sample Data and Our Linear Regression Model'); plt.xlabel(xlabel); plt.ylabel(ylabel)\n",
    "plt.legend()\n",
    "plt.grid(True); plt.xlim([1.0,2.4]); plt.ylim([5,20])\n",
    "\n",
    "for ssplope in range(0.0,12.0,0.5):\n",
    "    \n",
    "    sayhat = linear.intercept + sslope*X\n",
    "    sdif_mean = np.average(y) - np.average(sayhat)\n",
    "    ssint = linear.intercept + sdif_mean\n",
    "    ssayhat2 =ssint + aslope*X\n",
    "    mse[i] = np.linalg.norm(ssayhat2 - y)\n",
    "    \n",
    "\n",
    "\n",
    "plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.2, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visualize the Linear Regression Model\n",
    "\n",
    "Now let's visualize the linear regression model with the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The model parameters are, slope (b1) = -9.1, and the intercept (b0) = 28.35\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 68676 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('The model parameters are, slope (b1) = ' + str(round(linear.slope,2)) + ', and the intercept (b0) = ' + str(round(linear.intercept,2)))\n",
    "plt.scatter(X, y, color = 'darkorange',edgecolor='black',alpha=0.8,label='sample data',zorder=10)\n",
    "plt.plot(dX, linear.intercept + linear.slope*dX, 'black', label='linear regression model',zorder=1)\n",
    "plt.title('Sample Data and Linear Regression Model'); plt.xlabel(xlabel); plt.ylabel(ylabel)\n",
    "plt.legend(); plt.grid(); plt.ylim([ymin,ymax]); plt.xlim([xmin,xmax])\n",
    "plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.4, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model Confidence Intervals\n",
    "\n",
    "Let's calculate the 95% confidence interval for the linear regression model slope parameter, $b_1$, of our model.\n",
    "\n",
    "\\begin{equation}\n",
    "CI_{\\hat{b_1}}: \\hat{b}_1 ± t_{(\\frac{\\alpha}{2},n-2)} \\times SE_{b_1} \n",
    "\\end{equation}\n",
    "\n",
    "\\begin{equation}\n",
    "CI_{\\hat{b_0}}: \\hat{b}_0 ± t_{(\\frac{\\alpha}{2},n-2)} \\times SE_{b_0} \n",
    "\\end{equation}\n",
    "\n",
    "where standard errors are calculated as:\n",
    "\n",
    "\\begin{equation}\n",
    "SE_{b_1} = \\frac{\\sqrt{n}\\hat{\\sigma}}{\\sqrt{n-1} \\sqrt{\\sum(x_i - \\overline{x})^2  } }\n",
    "\\end{equation}\n",
    "\n",
    "\\begin{equation}\n",
    "SE_{b_0} = \\sqrt{ \\frac{\\hat{\\sigma}^2}{n-2} }\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The model parameters confidence intervals at a 0.95 significance level are:\n",
      "Slope: P2.5 = -10.26 , P97.5 = -7.94\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 108724 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "alpha = 0.05\n",
    "tstat = st.t.ppf([alpha/2,1-alpha/2], len(X)-2)            # calculate t-stat for confidence interval\n",
    "slope_lower,slope_upper = linear.slope + tstat*linear.stderr # calculate the lower and upper confidence interval for b1\n",
    "#intercept_lower,intercept_upper = linear.intercept + tstat*linear.intercept_stderr # \"\" for b0\n",
    "\n",
    "print('The model parameters confidence intervals at a ' + str(1-alpha) + ' significance level are:')\n",
    "print('Slope: P' + str(alpha/2*100) + ' = '+ str(round(slope_lower,2)) + ' , P' + str((1-alpha/2)*100) + ' = ' + str(round(slope_upper,2)))\n",
    "#print('Intercept: ' + str(round(intercept_lower,2)) + ' , ' + str(round(intercept_upper,2)))\n",
    "\n",
    "plt.scatter(X, y, color = 'darkorange',edgecolor='black',alpha=0.8,label='sample data',zorder=10)\n",
    "plt.plot(dX, linear.intercept + linear.slope*dX, 'black', label='linear regression model')\n",
    "plt.plot(dX, linear.intercept + slope_upper*dX, 'black',ls='--',lw=1,label=r'alpha = ' + str(alpha) + ' confidence interval')\n",
    "plt.plot(dX, linear.intercept + slope_lower*dX, 'black',ls='--',lw=1)\n",
    "plt.fill_between(dX,linear.intercept + slope_upper*dX,linear.intercept + slope_lower*dX,color='red',alpha=0.3,zorder=1)\n",
    "plt.title('Sample Data, Linear Regression Model and Slope Confidence Intervals'); plt.xlabel(r'Density ($g/cm^3$)'); plt.ylabel('Porosity (%)')\n",
    "plt.legend(loc='lower left'); plt.grid(); plt.ylim([ymin,ymax]); plt.xlim([xmin,xmax])\n",
    "plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.4, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model Prediction Intervals\n",
    "\n",
    "Let's calculate the prediction intervals.\n",
    "\n",
    "\\begin{equation}\n",
    "\\hat{y}_{n+1} ± t_{(\\frac{\\alpha}{2},n-2)} \\sqrt{MSE}\\ \\times \\sqrt{1+\\frac{1}{n}+\\frac{(x_{n+1}-\\overline{x})^2}{\\sum_{i=1}^{n}(x_{i}-\\overline{x})^2}  }\n",
    "\\end{equation}\n",
    "\n",
    "Note, this is the standard error of the prediction:\n",
    "\n",
    "\\begin{equation}\n",
    "SE_{\\hat{y}_{n+1}} = \\sqrt{MSE}\\ \\times \\sqrt{1+\\frac{1}{n}+\\frac{(x_{n+1}-\\overline{x})^2}{\\sum_{i=1}^{n}(x_{i}-\\overline{x})^2}  }\n",
    "\\end{equation}\n",
    "\n",
    "where MSE, model mean square error calculated as:\n",
    "\n",
    "\\begin{equation}\n",
    "MSE = \\sum_{i=1}^n\\frac{(y_i - \\hat{y}_i)^2}{n-2} = \\sum_{i=1}^n \\frac{\\left(y_i - (b_1 x - b_0) \\right)^2}{n-2}\n",
    "\\end{equation}\n",
    "\n",
    "Note, that this indicates that prediction intervals are wider the further we estimate from the mean of the predictor feature values. We can substitute model MSE, MSE, and standard error of the estimate, $SE_{\\hat{y}_{n+1}}$ for the final form is:\n",
    "\n",
    "\\begin{equation}\n",
    "\\hat{y}_{n+1} ± t_{(\\frac{\\alpha}{2},n-2)} \\sqrt{\\sum_{i=1}^n \\frac{\\left(y_i - (b_1 x - b_0) \\right)^2}{n-2}}\\sqrt{1+\\frac{1}{n}+\\frac{(x_{n+1}-\\overline{x})^2}{\\sum_{i=1}^{n}(x_{i}-\\overline{x})^2}  }\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "For the alpha level of: alpha = 0.05, \n",
      "the t-statistic is: -1.98\n",
      "\n",
      "For the prediction at: X = 2.0, \n",
      "the standard error of the prediction is: 1.73\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 85144 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "new_X = 2.0\n",
    "alpha = 0.05\n",
    "\n",
    "tstat = st.t.ppf([alpha/2,1-alpha/2], len(X)-2)\n",
    "\n",
    "print('For the alpha level of: alpha = ' + str(alpha) + \\\n",
    "      ', \\nthe t-statistic is: ' + str(round(tstat[0],2)))\n",
    "\n",
    "yhat = linear.intercept + linear.slope*X\n",
    "MSE = np.sum(np.power(y-yhat,2))/(len(y)-2) # mean square error\n",
    "est_stderr = math.sqrt(MSE) \\\n",
    "      *math.sqrt(1 + 1/len(y) + np.power(new_X - np.average(X),2)/ \\\n",
    "      np.sum(np.power(X-np.average(X),2)))\n",
    "\n",
    "print('\\nFor the prediction at: X = ' + str(new_X) + \\\n",
    "      ', \\nthe standard error of the prediction is: ' + str(round(est_stderr,2)))\n",
    "\n",
    "y_pred_lower, y_pred_upper = linear.intercept + linear.slope*new_X + tstat*est_stderr\n",
    "\n",
    "plt.scatter(X, y, color = 'darkorange',edgecolor='black',alpha=0.8,label='sample data',zorder=1)\n",
    "plt.plot(dX, linear.intercept + linear.slope*dX, 'black', label='linear regression model',zorder=1)\n",
    "plt.scatter(new_X, linear.intercept + linear.slope*new_X,s=80,color='yellow',edgecolor='black',label='prediction, $\\hat{y}$',zorder=2)\n",
    "plt.plot([new_X,new_X],[y_pred_lower,y_pred_upper],color='black',linestyle='dashed',zorder=1,label='prediction interval')\n",
    "plt.title('Sample Data, Linear Regression Model and Prediction Interval'); plt.xlabel(r'Density ($g/cm^3$)'); plt.ylabel('Porosity (%)')\n",
    "plt.legend(); plt.grid(); plt.ylim([4,22]); plt.xlim([1.0,2.4])\n",
    "plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.4, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot the prediction interval over the entire range of possible predictions and compare with the confidence interval.\n",
    "\n",
    "* we will also plot the standard error of the estimate to show the subtle heteroscedasticity of the prediction interval, the prediction error increases away from the mean of predictor feature values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 161492 characters]",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "alpha = 0.05\n",
    "\n",
    "tstat = st.t.ppf([alpha/2,1-alpha/2], len(X)-2)   \n",
    "slope_lower,slope_upper = linear.slope + tstat*linear.stderr  # confidence interval\n",
    "\n",
    "yhat = linear.intercept + linear.slope*X                 \n",
    "MSE = np.sum(np.power(y-yhat,2))/(len(X)-2)            # prediction interval: model MSE\n",
    "    \n",
    "y_pred_lower_mat = np.zeros(len(dX)); y_pred_upper_mat = np.zeros(len(dX))\n",
    "est_stderr_mat = np.zeros(len(dX))\n",
    "\n",
    "for inew_X, new_X in enumerate(dX):\n",
    "    est_stderr_mat[inew_X] = math.sqrt(MSE) * math.sqrt(1 + 1/len(X) + np.power(new_X - np.average(X),2)/np.sum(np.power(X-np.average(X),2)))\n",
    "    y_pred_lower_mat[inew_X], y_pred_upper_mat[inew_X] = linear.intercept + linear.slope*new_X + tstat*est_stderr_mat[inew_X]\n",
    "\n",
    "plt.subplot(121)\n",
    "plt.plot(dX,est_stderr_mat,c='r',ls='--',lw=1,label='standard error of the prediction')\n",
    "plt.plot([np.average(X),np.average(X)],[1.7,1.8],c='black',lw=1)\n",
    "plt.annotate('Prediction Feature Mean',[np.average(dX)*0.98,1.767],rotation=90)\n",
    "plt.title('Standard Error of the Estimate vs. Predictor Value'); plt.xlabel(r'Density ($g/cm^3$)'); plt.ylabel('Standard Error of the Prediction, Porosity (%)')\n",
    "plt.legend(loc='lower left'); plt.grid(); plt.xlim([xmin,xmax]); plt.ylim([1.7,1.8])\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.plot(dX, linear.intercept + linear.slope*dX, 'black', label='linear regression model')\n",
    "plt.plot([np.average(X),np.average(X)],[ymin,ymax],c='black',lw=1)\n",
    "plt.annotate('Prediction Feature Mean',[np.average(X)*0.98,16],rotation=90)\n",
    "plt.plot(dX,y_pred_lower_mat,c='black',lw=1,zorder=1)\n",
    "plt.plot(dX,y_pred_upper_mat,c='black',lw=1)\n",
    "plt.fill_between(dX,y_pred_upper_mat,y_pred_lower_mat,color='red',alpha=0.3,zorder=1,label=r'alpha = ' + str(alpha) + ' prediction interval')\n",
    "\n",
    "plt.plot(dX, linear.intercept + slope_upper*dX, 'black',lw=1,ls='-.')\n",
    "plt.plot(dX, linear.intercept + slope_lower*dX, 'black',lw=1,ls='-.')\n",
    "plt.fill_between(dX,linear.intercept + slope_upper*dX,linear.intercept + slope_lower*dX,color='black',alpha=0.3,zorder=1,label=r'alpha = ' + str(alpha) + ' confidence interval')\n",
    "plt.title('Linear Regression Model, Confidence Interval and Prediction Interval'); plt.xlabel(r'Density ($g/cm^3$)'); plt.ylabel('Porosity (%)')\n",
    "plt.legend(); plt.grid(); plt.ylim([ymin,ymax]); plt.xlim([xmin,xmax])\n",
    "\n",
    "plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=1.4, wspace=0.2, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model Checking\n",
    "\n",
    "Let's test the slope with the following hypothesis test:\n",
    "\n",
    "\\begin{equation}\n",
    "H_0: b_{1} = 0.0\n",
    "\\end{equation}\n",
    "\n",
    "\\begin{equation}\n",
    "H_1: b_{1} \\ne 0.0\n",
    "\\end{equation}\n",
    "\n",
    "and see if we can reject this hypothesis, $H_{0}$ , that the slope parameter is equal to 0.0.  If we reject this null hypothesis, we show that the slope is meaning full and there is a linear relationship between density and porosity that we can use.\n",
    "\n",
    "Fortunately, the linregress function from the stats module of SciPy package provides us with the two sided p-value for this test.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The linear regression model slope parameter p-value is 0.0.\n"
     ]
    }
   ],
   "source": [
    "print('The linear regression model slope parameter p-value is ' + str(round(linear.pvalue,20)) + '.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's calculate the $r^2$, proportion of variance explained. Here's the variance explained by the model:\n",
    "\n",
    "\\begin{equation}\n",
    "𝑠𝑠𝑟𝑒𝑔 = \\sum_{𝑖=1}^{𝑛}\\left(\\hat{y}_i - \\overline{y}\\right)^2 \n",
    "\\end{equation}\n",
    "\n",
    "and the variance not explained by the model:\n",
    "\n",
    "\\begin{equation}\n",
    "𝑠𝑠𝑟𝑒sid = \\sum_{𝑖=1}^{𝑛}\\left(y_i - \\hat{y}\\right)^2 \n",
    "\\end{equation}\n",
    "\n",
    "Now we can calculate the variance explained as:\n",
    "\n",
    "\\begin{equation}\n",
    "𝑟^2 = \\frac{𝑠𝑠𝑟𝑒𝑔}{𝑠𝑠𝑟𝑒𝑔+𝑠𝑠𝑟𝑒𝑠𝑖𝑑} = \\frac{\\text{variance explained}}{\\text{total variance}}\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The r-squared value of the linear regression model is 0.702\n"
     ]
    }
   ],
   "source": [
    "r2 = linear.rvalue**2\n",
    "print('The r-squared value of the linear regression model is ' + str(round(r2,3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's perform a f-test for the significance of all model parameters at once.\n",
    "\n",
    "\\begin{equation}\n",
    "H_0: b_i = 0, \\forall i\n",
    "\\end{equation}\n",
    "\n",
    "\\begin{equation}\n",
    "H_1: otherwise\n",
    "\\end{equation}\n",
    "\n",
    "To test this hypothesis we calculate the $f_{statistic}$ and $f_{critical}$ as:\n",
    "\n",
    "\\begin{equation}\n",
    "f_{statistic} = \\frac{\\text{Mean Squares of Model}}{\\text{Mean Squares of Error}}   \n",
    "\\end{equation}\n",
    "\n",
    "\\begin{equation}\n",
    "f_{statistic} = \\frac{ \\frac{\\sum_{𝑖=1}^{𝑛}(\\hat{y}_i - \\overline{y})^2}{k-1} } { \\frac{\\sum_{𝑖=1}^{𝑛}\\left(y_i - \\hat{y}\\right)^2}{n-k} }\n",
    "\\end{equation}\n",
    "\n",
    "and now we can calculate the $f_{critical}$ value as:\n",
    "\n",
    "\\begin{equation}\n",
    "𝑓_{𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙}  = f_{k-1,n-k}(1-𝛼)\n",
    "\\end{equation}\n",
    "\n",
    "Since this is a one-tail test, if $f_{statistic} > f_{critical}$, then we reject the null hypothesis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The f-stat is : 345.68 and the f-critical is : 3.933\n",
      "Therefore we reject the null hypothesis, \n",
      "our model parameters are significant\n"
     ]
    }
   ],
   "source": [
    "alpha = 0.05                                                 # alpha level, 1 - confidence level\n",
    "k = 2                                                        # number of model parameters\n",
    "ms_model = np.sum(np.power(y-np.average(y),2))/(k-1)     # f-test, calculate f-stat and f-critical\n",
    "ms_error = np.sum(np.power(y-yhat,2))/(len(y)-k)\n",
    "f_stat = ms_model/ms_error                                   \n",
    "f_crit = st.f.ppf(1-alpha, k-1, len(y)-k)\n",
    "\n",
    "print('The f-stat is : ' + str(round(f_stat,2)) + ' and the f-critical is : ' + str(round(f_crit,3)))\n",
    "\n",
    "if f_stat > f_crit:\n",
    "    print('Therefore we reject the null hypothesis, \\nour model parameters are significant')\n",
    "if f_stat <= f_crit:\n",
    "    print('Therefore we fail to reject the null hypothesis, \\nour model parameters are NOT significant')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Correlation Coefficient\n",
    "\n",
    "We can also observe correlation coefficient.\n",
    "\n",
    "* valid and useful for our linear model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The correlation coefficient is = -0.84\n"
     ]
    }
   ],
   "source": [
    "print('The correlation coefficient is = ' + str(round(linear.rvalue,2))) # correlation coefficient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model Cross Validation\n",
    "\n",
    "Let's use this model to make a prediction at all the data locations.  \n",
    "\n",
    "* now plot a standard model cross validation plot, actual vs. predicted values for the response feature\n",
    "* note, we are only looking at data used to build the model, known as training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 64836 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "alpha = 0.05                                                 # alpha level, 1 - confidence level\n",
    "yhat = linear.slope * X + linear.intercept              # estimate at the training data location\n",
    "plt.scatter(yhat,y,color='darkorange',alpha=0.8,edgecolor='black')\n",
    "plt.plot([ymin,ymax],[ymin,ymax],color='black',linewidth=1)\n",
    "plt.ylabel('Actual ' + ylabel ); plt.xlabel('Estimated ' + ylabel ); \n",
    "plt.title('Training Data Cross Validation Plot')\n",
    "plt.xlim(ymin,ymax); plt.ylim(ymin,ymax)\n",
    "plt.grid(); plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.4, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's look at the distribution of estimates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 42116 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(yhat,color='darkorange',alpha=0.4,edgecolor='black',bins=np.linspace(ymin,ymax,30),label='Predictions')\n",
    "plt.hist(y,color='grey',alpha=0.4,edgecolor='black',bins=np.linspace(ymin,ymax,30),label='Data')\n",
    "\n",
    "plt.annotate('Data Mean = ' + str(np.round(np.average(y),2)),xy=[17.5,20],color='grey')\n",
    "plt.annotate('Predictions Mean = ' + str(np.round(np.average(yhat),2)),xy=[17.5,19.0],color='darkorange')\n",
    "\n",
    "plt.annotate('Data St.Dev. = ' + str(np.round(np.std(y),2)),xy=[17.5,18],color='grey')\n",
    "plt.annotate('Predictions St.Dev. = ' + str(np.round(np.std(yhat),2)),xy=[17.5,17.0],color='darkorange')\n",
    "\n",
    "\n",
    "plt.title(yname + \" Predictions with Linear Model\")\n",
    "plt.xlabel(ylabel); plt.ylabel('Frequency'); plt.legend(loc='upper right')\n",
    "plt.xlim([0,25]); plt.ylim([0,25])\n",
    "plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is useful to plot the predictions of porosity and porosity data vs. the density data. From this plot we can observe the linear limitation of our model and get a sense of the unexplained variance $\\frac{\\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2} {n-1}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 102860 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X, y,color='black',alpha=0.8,facecolor='grey',label='sample data',zorder=10)\n",
    "plt.scatter(X, yhat,color='black',s=40,facecolor='darkorange',alpha=0.8,label='model',zorder=10)\n",
    "plt.plot(dX, linear.intercept + linear.slope*dX, 'black', label='linear regression model',zorder=1)\n",
    "\n",
    "for i in range(0,len(X)):\n",
    "    plt.plot([X[i],X[i]],[y[i],yhat[i]],c='grey',alpha=0.8,ls='--',zorder=1)\n",
    "plt.title('Sample Data and Model'); plt.xlabel(xlabel); plt.ylabel(ylabel)\n",
    "plt.legend()\n",
    "plt.grid(True); plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next let's calculate the residual and check their distribution. \n",
    "\n",
    "* residuals are the true values at the data locations minus the estimates at the data locations, $y_i - \\hat{y}_i$.  \n",
    "\n",
    "We want to make sure the average is close to 0.0 (unbiased estimates) and to observe the shape and spread of the residual distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The average of the residuals is 0.0\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 20724 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "residual = y - yhat\n",
    "plt.hist(residual,color='darkorange',alpha=0.8,edgecolor='black',bins=np.linspace(-5,5,30))\n",
    "plt.title(\"Residual\")\n",
    "plt.xlabel(yname + ' True - Estimate (' + yunits +')')\n",
    "plt.grid(True)\n",
    "print('The average of the residuals is ' + str(round(np.mean(residual),2)))\n",
    "plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will check the residual vs. the fitted value.  \n",
    "\n",
    "* we want to see if the errors are consistent over the range of fitted values.  \n",
    "\n",
    "* for example, we could use this plot to identify higher error or systematic under- or overestimation over a specific range of fitted values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 44664 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(yhat,residual,facecolor='darkorange',alpha=0.8,color='black')\n",
    "plt.title('Residual vs. Fitted Value')\n",
    "plt.xlabel(yname + ' Estimate (%)')\n",
    "plt.ylabel(yname + ' Residual (%)')\n",
    "plt.plot([5,20], [0,0],'black')\n",
    "plt.grid(True); plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let's Rebuild the Model with scikit-learn\n",
    "\n",
    "* scikit-learn is one of the most popular Python machine learning packages\n",
    "\n",
    "* demonstrates the standard scikit-learn approach\n",
    "\n",
    "1. **instantiate** - make the model object while setting the hyperparameters (none for linear regression)\n",
    "2. **fit** - train the model parameters with training data (we aren't doing train and test split yet) \n",
    "3. **predict** - use the trained model to make predictions (for demonstration we just predict at the training data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 104320 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "linear_model = LinearRegression()                           # instantiate the prediction model\n",
    "linear_model.fit(X.reshape(-1, 1),y)             # train the model with the training data\n",
    "yhat_scikit = linear_model.predict(X.reshape(-1, 1)) #$ predict with the trained model\n",
    "\n",
    "plt.scatter(X, y,color='black',alpha=0.8,facecolor='grey',label='sample data',zorder=10)\n",
    "plt.scatter(X, yhat_scikit,color='black',s=40,facecolor='darkorange',alpha=0.8,label='model',zorder=10)\n",
    "plt.plot(dX, linear.intercept + linear.slope*dX, 'black', label='linear regression model',zorder=1)\n",
    "\n",
    "for i in range(0,len(X)):\n",
    "    plt.plot([X[i],X[i]],[y[i],yhat_scikit[i]],c='grey',alpha=0.8,ls='--',zorder=1)\n",
    "    \n",
    "plt.title('Sample Data and scikit-learn Model'); plt.xlabel(xlabel); plt.ylabel(ylabel)\n",
    "plt.legend()\n",
    "plt.grid(True); plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.1, hspace=0.2); plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Comments\n",
    "\n",
    "This was a basic demonstration of linear regression in Python with the SciPy package.\n",
    "\n",
    "I have other demonstrations on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy. \n",
    "  \n",
    "I hope this was helpful,\n",
    "\n",
    "*Michael*\n",
    "\n",
    "#### The Author:\n",
    "\n",
    "### Michael Pyrcz, Professor, The University of Texas at Austin \n",
    "*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*\n",
    "\n",
    "With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development. \n",
    "\n",
    "For more about Michael check out these links:\n",
    "\n",
    "#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)\n",
    "\n",
    "#### Want to Work Together?\n",
    "\n",
    "I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.\n",
    "\n",
    "* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you! \n",
    "\n",
    "* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!\n",
    "\n",
    "* I can be reached at mpyrcz@austin.utexas.edu.\n",
    "\n",
    "I'm always happy to discuss,\n",
    "\n",
    "*Michael*\n",
    "\n",
    "Michael Pyrcz, Ph.D., P.Eng. Professor, Cockrell School of Engineering and The Jackson School of Geosciences, The University of Texas at Austin\n",
    "\n",
    "#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
