{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<p align=\"center\">\n",
    "    <img src=\"https://github.com/GeostatsGuy/GeostatsPy/blob/master/TCG_color_logo.png?raw=true\" width=\"220\" height=\"240\" />\n",
    "\n",
    "</p>\n",
    "\n",
    "## Interactive Spatial Data Declustering Demonstration\n",
    "\n",
    "\n",
    "### Michael Pyrcz, Associate Professor, University of Texas at Austin \n",
    "\n",
    "##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)\n",
    "\n",
    "\n",
    "### The Interactive Workflow\n",
    "\n",
    "Here's a simple workflow for calculating and evaluating spatial data declustering weights smapling bias due to clustered sampled mitigation.\n",
    "\n",
    "\n",
    "### Basic Univariate Summary Statistics and Data Distribution Representativity Plotting in Python with GeostatsPy\n",
    "\n",
    "Here's a simple workflow with some basic univariate statistics and distribution representativity. This should help you get started data declustering to address spatial sampling bias.\n",
    "\n",
    "#### Geostatistical Sampling Representativity\n",
    "\n",
    "In general, we should assume that all spatial data that we work with is biased.\n",
    "\n",
    "##### Source of Spatial Sampling Bias\n",
    "\n",
    "Data is collected to answer questions:\n",
    "* how far does the contaminant plume extend? – sample peripheries\n",
    "* where is the fault? – drill based on seismic interpretation\n",
    "* what is the highest mineral grade? – sample the best part\n",
    "* who far does the reservoir extend? – offset drilling\n",
    "and to maximize NPV directly:\n",
    "* maximize production rates\n",
    "\n",
    "**Random Sampling**: when every item in the population has a equal chance of being chosen. Selection of every item is independent of every other selection.\n",
    "Is random sampling sufficient for subsurface?  Is it available?\n",
    "* it is not usually available, would not be economic\n",
    "* data is collected answer questions\n",
    "    * how large is the reservoir, what is the thickest part of the reservoir \n",
    "* and wells are located to maximize future production\n",
    "    * dual purpose appraisal and injection / production wells!\n",
    "\n",
    "**Regular Sampling**: when samples are taken at regular intervals (equally spaced).  \n",
    "* less reliable than random sampling.\n",
    "* Warning: may resonate with some unsuspected environmental variable.\n",
    "\n",
    "What do we have?\n",
    "* we usually have biased, opportunity sampling \n",
    "* we must account for bias (debiasing will be discussed later)\n",
    "\n",
    "So if we were designing sampling for representativity of the sample set and resulting sample statistics, by theory we have 2 options, random sampling and regular sampling.\n",
    "\n",
    "* What would happen if you proposed random sampling in the Gulf of Mexico at $150M per well?\n",
    "\n",
    "We should not change current sampling methods as they result in best economics, we should address sampling bias in the data.\n",
    "\n",
    "Never use raw spatial data without access sampling bias / correcting.\n",
    "\n",
    "##### Mitigating Sampling Bias\n",
    "\n",
    "In this demonstration we will take a biased spatial sample data set and apply declustering using **GeostatsPy** functionality.\n",
    "\n",
    "#### Objective \n",
    "\n",
    "In the PGE 383: Stochastic Subsurface Modeling class I want to provide hands-on experience with building subsurface modeling workflows. Python provides an excellent vehicle to accomplish this. I have coded a package called GeostatsPy with GSLIB: Geostatistical Library (Deutsch and Journel, 1998) functionality that provides basic building blocks for building subsurface modeling workflows. \n",
    "\n",
    "The objective is to remove the hurdles of subsurface modeling workflow construction by providing building blocks and sufficient examples. This is not a coding class per se, but we need the ability to 'script' workflows working with numerical methods.    \n",
    "\n",
    "#### Getting Started\n",
    "\n",
    "Here's the steps to get setup in Python with the GeostatsPy package:\n",
    "\n",
    "1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/). \n",
    "2. From Anaconda Navigator (within Anaconda3 group), go to the environment tab, click on base (root) green arrow and open a terminal. \n",
    "3. In the terminal type: pip install geostatspy. \n",
    "4. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality. \n",
    "\n",
    "You will need to copy the data file to your working directory.  They are available here:\n",
    "\n",
    "* Tabular data - sample_data_biased.csv at https://git.io/fh0CW\n",
    "\n",
    "There are exampled below with these functions. You can go here to see a list of the available functions, https://git.io/fh4eX, other example workflows and source code. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import geostatspy.GSLIB as GSLIB          # GSLIB utilies, visualization and wrapper\n",
    "import geostatspy.geostats as geostats    # GSLIB methods convert to Python        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also need some standard packages. These should have been installed with Anaconda 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np                        # ndarrys for gridded data\n",
    "import pandas as pd                       # DataFrames for tabular data\n",
    "import os                                 # set working directory, run executables\n",
    "import matplotlib.pyplot as plt           # for plotting\n",
    "from scipy import stats                   # summary statistics\n",
    "from ipywidgets import interactive        # widgets and interactivity\n",
    "from ipywidgets import widgets                            \n",
    "from ipywidgets import Layout\n",
    "from ipywidgets import Label\n",
    "from ipywidgets import VBox, HBox\n",
    "from matplotlib.ticker import (MultipleLocator, AutoMinorLocator) # control of axes ticks\n",
    "plt.rc('axes', axisbelow=True)            # set axes and grids in the background for all plots\n",
    "from matplotlib.patches import Rectangle  # drawing shapes on plots\n",
    "from statsmodels.stats.weightstats import DescrStatsW"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.  \n",
    "\n",
    "#### Declare Functions\n",
    "\n",
    "These functions read in the multiple realizations and produce local statistical summaries that we will cover below.  They will shortly be added to GeostatsPy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def declus(df, xcol, ycol, noff, csize):\n",
    "    # modified from GeostatsPy Python Package to run 1 cell size and report all cell offsets\n",
    "    # Load data and set up arrays\n",
    "    nd = len(df)\n",
    "    x = df[xcol].values\n",
    "    y = df[ycol].values\n",
    "    wt = np.zeros([nd,noff])\n",
    "    wtopt = np.ones(nd)\n",
    "    index = np.zeros(nd, np.int32)\n",
    "    roff = float(noff)\n",
    "\n",
    "    # Calculate data extents\n",
    "    xmin = np.min(x); xmax = np.max(x); ymin = np.min(y); ymax = np.max(y)\n",
    "    xmin = 0.0; ymin = 0.0; xmax = 1000.0; ymax = 1000.0\n",
    "    # Define a \"lower\" origin to use for the cell sizes\n",
    "    xo1 = xmin\n",
    "    yo1 = ymin\n",
    "\n",
    "    # Only 1 cell size\n",
    "    ncellx = int((xmax - (xo1 - csize)) / csize) + 1\n",
    "    ncelly = int((ymax - (yo1 - csize)) / csize) + 1\n",
    "    ncellt = ncellx * ncelly\n",
    "    cellwt = np.zeros(ncellt)\n",
    "    \n",
    "    # Loop over all the origin offsets selected\n",
    "    xfac = min((csize / roff), (0.5 * (xmax - xmin)))\n",
    "    yfac = min((csize / roff), (0.5 * (ymax - ymin)))\n",
    "    for kp in range(0, noff ):\n",
    "        xo = xo1 - (float(kp)) * xfac\n",
    "        yo = yo1 - (float(kp)) * yfac\n",
    "\n",
    "        # Initialize the cumulative weight indicators\n",
    "        cellwt.fill(0.0)\n",
    "\n",
    "        # Determine which cell each datum is in\n",
    "        for i in range(0, nd):\n",
    "            icellx = int((x[i] - xo) / csize) + 1\n",
    "            icelly = int((y[i] - yo) / csize) + 1\n",
    "            icell = icellx + (icelly - 1) * ncellx - 1\n",
    "            index[i] = icell\n",
    "            cellwt[icell] = cellwt[icell] + 1.0\n",
    "\n",
    "        # The weight assigned to each datum is inversely proportional to the\n",
    "        # number of data in the cell. We first need to get the sum of\n",
    "        # weights so that we can normalize the weights to sum to one\n",
    "        sumw = 0.0\n",
    "        for i in range(0, nd):\n",
    "            ipoint = index[i]\n",
    "            sumw = sumw + (1.0 / cellwt[ipoint])\n",
    "        sumw = 1.0 / sumw\n",
    "\n",
    "        # Accumulate the array of weights (that now sum to one)\n",
    "        for i in range(0, nd):\n",
    "            ipoint = index[i]\n",
    "            wt[i,kp] = (1.0 / cellwt[ipoint]) * sumw\n",
    "\n",
    "    # End loop over all offsets\n",
    "\n",
    "    # Correct weights over all offsets\n",
    "    for ioff in range(0, noff):\n",
    "        sumw = np.sum(wt[:,ioff])\n",
    "        wt[:,ioff] = wt[:,ioff]/sumw\n",
    "                        \n",
    "    return wt\n",
    "\n",
    "def weighted_percentile(data,perc,weights=None): # from Luca Jokull (https://stackoverflow.com/questions/21844024/weighted-percentile-using-numpy)\n",
    "    if weights is None:\n",
    "        return nanpercentile(data,perc)\n",
    "    else:\n",
    "        d=data[(~np.isnan(data))&(~np.isnan(weights))]\n",
    "        ix=np.argsort(d)\n",
    "        d=d[ix]\n",
    "        wei=weights[ix]\n",
    "        wei_cum=100.*np.cumsum(wei*1./sum(wei))\n",
    "        return np.interp(perc,wei_cum,d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the working directory\n",
    "\n",
    "I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#os.chdir(\"c:/PGE383\")             # set the working directory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Loading Tabular Data\n",
    "\n",
    "Here's the command to load our comma delimited data file in to a Pandas' DataFrame object.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#df = pd.read_csv('sample_data_biased.csv')     # load our data\n",
    "df = pd.read_csv('https://raw.githubusercontent.com/GeostatsGuy/GeoDataSets/master/sample_data_biased.csv')\n",
    "df['Porosity'] = df['Porosity'] * 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We loaded our file into our DataFrame called 'df'. But how do you really know that it worked? Visualizing the DataFrame would be useful and we already leard about these methods in this demo (https://git.io/fNgRW). \n",
    "\n",
    "#### Resample from the Data\n",
    "\n",
    "Option to reduce the number of spatial sample data to explore the impact on spatial uncertainty."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>Facies</th>\n",
       "      <th>Porosity</th>\n",
       "      <th>Perm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>14</td>\n",
       "      <td>400</td>\n",
       "      <td>900</td>\n",
       "      <td>1</td>\n",
       "      <td>22.366071</td>\n",
       "      <td>2372.383732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>246</td>\n",
       "      <td>730</td>\n",
       "      <td>169</td>\n",
       "      <td>1</td>\n",
       "      <td>13.372385</td>\n",
       "      <td>7.350097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>221</td>\n",
       "      <td>490</td>\n",
       "      <td>959</td>\n",
       "      <td>1</td>\n",
       "      <td>19.656596</td>\n",
       "      <td>403.008632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>136</td>\n",
       "      <td>0</td>\n",
       "      <td>309</td>\n",
       "      <td>0</td>\n",
       "      <td>8.777591</td>\n",
       "      <td>0.665880</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>31</td>\n",
       "      <td>800</td>\n",
       "      <td>300</td>\n",
       "      <td>1</td>\n",
       "      <td>14.708393</td>\n",
       "      <td>25.729277</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index    X    Y  Facies   Porosity         Perm\n",
       "0     14  400  900       1  22.366071  2372.383732\n",
       "1    246  730  169       1  13.372385     7.350097\n",
       "2    221  490  959       1  19.656596   403.008632\n",
       "3    136    0  309       0   8.777591     0.665880\n",
       "4     31  800  300       1  14.708393    25.729277"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seed = 73061\n",
    "ndata = 50\n",
    "df = df.sample(ndata,replace = False,random_state = seed) # extract 50 samples\n",
    "df = df.reset_index()                                     # reset the record index\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Summary Statistics for Tabular Data\n",
    "\n",
    "The table includes X and Y coordinates (meters), Facies 1 and 2 (1 is sandstone and 0 interbedded sand and mudstone), Porosity (fraction), and permeability as Perm (mDarcy). \n",
    "\n",
    "There are a lot of efficient methods to calculate summary statistics from tabular data in DataFrames. The describe command provides count, mean, minimum, maximum, and quartiles all in a nice data table. We use transpose just to flip the table so that features are on the rows and the statistics are on the columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>Facies</th>\n",
       "      <th>Porosity</th>\n",
       "      <th>Perm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>162.180000</td>\n",
       "      <td>481.000000</td>\n",
       "      <td>562.980000</td>\n",
       "      <td>0.86000</td>\n",
       "      <td>14.050819</td>\n",
       "      <td>223.418381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>84.369133</td>\n",
       "      <td>242.219752</td>\n",
       "      <td>308.571659</td>\n",
       "      <td>0.35051</td>\n",
       "      <td>3.873516</td>\n",
       "      <td>487.955880</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>14.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.854787</td>\n",
       "      <td>0.075819</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>97.000000</td>\n",
       "      <td>300.000000</td>\n",
       "      <td>302.250000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>11.088426</td>\n",
       "      <td>4.204787</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>163.500000</td>\n",
       "      <td>455.000000</td>\n",
       "      <td>609.500000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>13.378642</td>\n",
       "      <td>25.234794</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>239.250000</td>\n",
       "      <td>662.500000</td>\n",
       "      <td>877.750000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>15.880765</td>\n",
       "      <td>82.676408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>284.000000</td>\n",
       "      <td>920.000000</td>\n",
       "      <td>999.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>22.366071</td>\n",
       "      <td>2372.383732</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            index           X           Y    Facies   Porosity         Perm\n",
       "count   50.000000   50.000000   50.000000  50.00000  50.000000    50.000000\n",
       "mean   162.180000  481.000000  562.980000   0.86000  14.050819   223.418381\n",
       "std     84.369133  242.219752  308.571659   0.35051   3.873516   487.955880\n",
       "min     14.000000    0.000000    9.000000   0.00000   5.854787     0.075819\n",
       "25%     97.000000  300.000000  302.250000   1.00000  11.088426     4.204787\n",
       "50%    163.500000  455.000000  609.500000   1.00000  13.378642    25.234794\n",
       "75%    239.250000  662.500000  877.750000   1.00000  15.880765    82.676408\n",
       "max    284.000000  920.000000  999.000000   1.00000  22.366071  2372.383732"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Specify the Area of Interest\n",
    "\n",
    "It is natural to set the x and y coordinate and feature ranges manually. e.g. do you want your color bar to go from 0.05887 to 0.24230 exactly? Also, let's pick a color map for display. I heard that plasma is known to be friendly to the color blind as the color and intensity vary together (hope I got that right, it was an interesting Twitter conversation started by Matt Hall from Agile if I recall correctly). We will assume a study area of 0 to 1,000m in x and y and omit any data outside this area."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "xmin = 0.0; xmax = 1000.0               # range of x values\n",
    "ymin = 0.0; ymax = 1000.0               # range of y values\n",
    "vmin = 0.05; vmax = 25.0;               # range of porosity values\n",
    "cmap = plt.cm.inferno                   # color map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualizing Tabular Data with Location Maps¶ \n",
    "Let's try out locmap. This is a reimplementation of GSLIB's locmap program that uses matplotlib. I hope you find it simpler than matplotlib, if you want to get more advanced and build custom plots lock at the source. If you improve it, send me the new code. Any help is appreciated. To see the parameters, just type the command name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function geostatspy.GSLIB.locmap(df, xcol, ycol, vcol, xmin, xmax, ymin, ymax, vmin, vmax, title, xlabel, ylabel, vlabel, cmap, fig_name)>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GSLIB.locmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can populate the plotting parameters and visualize the porosity data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 56044 characters]",
      "text/plain": [
       "<Figure size 800x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x206ff614d50>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "GSLIB.locmap(df,'X','Y','Porosity',xmin,xmax,ymin,ymax,vmin,vmax,'Well Data - Porosity','X(m)','Y(m)','Porosity (fraction)',cmap,'locmap_Porosity')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look carefully, and you'll notice the the spatial samples are more dense in the high porosity regions and lower in the low porosity regions.  There is preferential sampling.  We cannot use the naive statistics to represent this region.  We have to correct for the clustering of the samples in the high porosity regions. \n",
    "\n",
    "Let's try cell declustering. We can interpret that we will want to minimize the declustering mean and that a cell size of between 100 - 200m is likely a good cell size, this is 'an ocular' estimate of the largest average spacing in the sparsely sampled regions.   \n",
    "\n",
    "Let's check out the declus program reimplimented from GSLIB."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "wmin = 0.0; wmax = 3.0\n",
    "noff = 3\n",
    "csize = 50.0\n",
    "\n",
    "# interactive calculation of the sample set (control of source parametric distribution and number of samples)\n",
    "style = {'description_width': 'initial'}\n",
    "l = widgets.Text(value='                                              Cell-based Declustering, Michael Pyrcz, Associate Professor, The University of Texas at Austin',layout=Layout(width='950px', height='30px'))\n",
    "csize = widgets.FloatSlider(min = 5.0, max = 1000, value = 100.0, step = 10.0, description = 'Cell Size',style=style,orientation='horizontal',\n",
    "                          layout=Layout(width='250px', height='50px'),continuous_update = False)\n",
    "csize.style.handle_color = 'gray'\n",
    "\n",
    "noff = widgets.IntSlider(min=1, max = 30, value = 5, step = 5, description = 'Number of Offsets',style=style,\n",
    "                        orientation='horizontal',layout=Layout(width='250px', height='50px'),continuous_update=False)\n",
    "noff.style.handle_color = 'gray'\n",
    "\n",
    "grid = widgets.Checkbox(value=True,description='Show Grid')\n",
    "\n",
    "uipars = widgets.HBox([csize,noff,grid],) \n",
    "ui = widgets.VBox([l,uipars],)\n",
    "\n",
    "def f_make_declus(csize,noff,grid,): # function to take parameters, make sample and plot\n",
    "    \n",
    "    wts =  declus(df,'X','Y',noff=noff,csize=csize)*len(df)\n",
    "    avg_wts = np.average(wts,axis=1)\n",
    "    \n",
    "    plt.subplot(131)\n",
    "    im = plt.scatter(df['X'],df['Y'],c=wts[:,0],cmap=cmap,vmin=wmin,vmax=wmax,alpha=0.8,linewidths=0.8,\n",
    "            edgecolors=\"black\",)\n",
    "    plt.title('Spatial Sample Data with Weights')\n",
    "    plt.xlim(xmin, xmax); plt.ylim(ymin, ymax); plt.xlabel('X (m)'); plt.ylabel('Y (m)')\n",
    "    cbar = plt.colorbar(im, orientation=\"vertical\", ticks=np.linspace(wmin, wmax, 10))\n",
    "    cbar.set_label('Weights',rotation=270, labelpad=20)\n",
    "     \n",
    "    if csize < 1000 and grid == True:\n",
    "        major_ticks = np.arange(0, 1001, csize)\n",
    "        plt.gca().set_xticks(major_ticks); plt.gca().set_yticks(major_ticks)\n",
    "        plt.grid()   \n",
    "\n",
    "    plt.subplot(132)\n",
    "    plt.hist(df['Porosity'],color='red',alpha=0.4,bins = np.linspace(vmin,vmax,20),edgecolor = 'black',label = 'Naive')\n",
    "    plt.hist(df['Porosity'],weights = avg_wts,color='gray',alpha=0.4,bins = np.linspace(vmin,vmax,20),edgecolor = 'black',label= 'Weighted')\n",
    "    plt.ylabel('Frequency'); plt.xlabel('Porosity (%)'); plt.title('Naive and Declustered Distributions')\n",
    "    plt.legend()\n",
    "    \n",
    "    plt.xlim(0,30); plt.ylim(0.0,int(len(df)*0.5))\n",
    "    \n",
    "    avg = np.average(df['Porosity'])\n",
    "    declus_avg = np.average(df['Porosity'],weights = avg_wts)\n",
    "    P10 = weighted_percentile(df['Porosity'],10,weights=np.ones(len(df)))\n",
    "    P90 = weighted_percentile(df['Porosity'],90,weights=np.ones(len(df)))\n",
    "    \n",
    "    plt.gca().add_patch(Rectangle((21.7,len(df)*0.345),7.8,len(df)*0.07,facecolor='white',edgecolor='black',linewidth=0.5)); \n",
    "    plt.text(22,len(df)*0.4,'Naive Mean   ' + str(round(avg,1))); \n",
    "    plt.text(22,len(df)*0.38,'Declus. Mean ' + str(round(declus_avg,1))); \n",
    "    plt.text(22,len(df)*0.36,'Correction    ' + str(round((declus_avg-avg)/avg*100,0)) + '%'); \n",
    "    \n",
    "    plt.gca().add_patch(Rectangle((21.7,len(df)*0.235),4.5,len(df)*0.08,facecolor='white',edgecolor='black',linewidth=0.5)); \n",
    "    plt.text(22,len(df)*0.3,'Mean ' + str(round(np.average(df['Porosity'],weights = avg_wts),1))); \n",
    "    plt.text(22,len(df)*0.28,'P90    ' + str(round(P90,1))); \n",
    "    plt.text(22,len(df)*0.26,'P10    ' + str(round(P10,1))); \n",
    "    plt.text(22,len(df)*0.24,'n        ' + str(len(df)))\n",
    "\n",
    "    plt.plot([avg,avg],[0.0,45],color = 'red')    \n",
    "    plt.plot([P10,P10],[0.0,45],color = 'black',linestyle='dashed')\n",
    "    plt.plot([declus_avg,declus_avg],[0.0,45],color = 'black')\n",
    "    plt.plot([P90,P90],[0.0,45],color = 'black',linestyle='dashed')\n",
    "    \n",
    "    plt.subplot(133)\n",
    "    plt.boxplot(wts.T)\n",
    "    plt.subplots_adjust(left=0.0, bottom=0.0, right=1.0, top=1.2, wspace=0.2, hspace=0.2)\n",
    "    for label in (plt.gca().get_xticklabels() + plt.gca().get_yticklabels()):\n",
    "        label.set_fontname('Arial')\n",
    "        label.set_fontsize(8)\n",
    "    plt.scatter(np.linspace(1,len(df),len(df)),avg_wts,color='blue',alpha=0.8,marker='x',s=50,linewidth=1,label='Average')\n",
    "    plt.plot([1,len(df)],[1.,1.],c='red')\n",
    "    plt.xlabel('Data Index'); plt.ylabel('Data Weight Over Cell Mesh Offsets'); plt.title('Weights Variation Over Random Cell Mesh Offsets')\n",
    "    plt.xlim(0,len(df)+1); #plt.ylim(0,1.5)\n",
    "    plt.legend(loc = 'lower right')\n",
    "    \n",
    "    plt.subplots_adjust(left=0.0, bottom=0.0, right=3.0, top=1.2, wspace=0.2, hspace=0.2)\n",
    "    plt.show()\n",
    "    \n",
    "# connect the function to make the samples and plot to the widgets    \n",
    "interactive_plot = widgets.interactive_output(f_make_declus, {'csize':csize, 'noff':noff, 'grid':grid})\n",
    "interactive_plot.clear_output(wait = True)               # reduce flickering by delaying plot updating\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interactive Cell-based Declustering Demostration\n",
    "\n",
    "* select the cell size and number of cell mesh offsets and visualize the declustering method \n",
    "\n",
    "#### Michael Pyrcz, Associate Professor, University of Texas at Austin \n",
    "\n",
    "##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)\n",
    "\n",
    "### The Inputs\n",
    "\n",
    "* **Cell Size**: the size the of the cells in the mesh, **Number of Offsets**: number of cell mesh offsets to average to calculate the data weights "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1dc2d081d7fb468fa86acca68df887d1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Text(value='                                              Cell-based Declustering, Michael Pyrc…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "43bab73de6294a25ab5ddbf61b524455",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output(outputs=({'output_type': 'display_data', 'data': {'text/plain': '<Figure size 640x480 with 4 Axes>', 'i…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(ui, interactive_plot)                            # display the interactive plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Comments\n",
    "\n",
    "This is an interactive demonstration of cell-based declustering to correct for sampling bias. Much more could be done, I have other demonstrations on the basics of working with DataFrames, ndarrays and many other workflows availble at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.\n",
    "\n",
    "I hope this was helpful,\n",
    "\n",
    "*Michael*\n",
    "\n",
    "Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin\n",
    "\n",
    "#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
