{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction: Interacting with Slack from Python\n",
    "\n",
    "In this notebook, we will use `Slacker` to interact with Slack. This allows us to send messages and upload files, which can be useful when training models, or monitoring scripts (or just for fun of course)!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First off, install the package. Documentation can be found [here](https://github.com/os/slacker). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install - q - U Slacker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ALl outputs \n",
    "from IPython.core.interactiveshell import InteractiveShell\n",
    "InteractiveShell.ast_node_interactivity = 'all'\n",
    "\n",
    "# Slacker\n",
    "from slacker import Slacker\n",
    "\n",
    "# Data manipulation\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "\n",
    "\n",
    "# Visualization\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.style.use('fivethirtyeight')\n",
    "\n",
    "# Displaying images\n",
    "from IPython.display import Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get a Slack Legacy API Token\n",
    "\n",
    "Most importantly, you need a [Slack workspace](https://slack.com/). I recommend making your own workspace where you are the admin just to get familiar with this package. Once you have your workspace, obtain a legacy token to enable interaction. Head to the following link to get a token.\n",
    "\n",
    "https://api.slack.com/custom-integrations/legacy-tokens\n",
    "\n",
    "![](images/legacy_tokens.PNG)\n",
    "\n",
    "Store the token in a safe location (not on GitHub)!\n",
    "\n",
    "This token will allow us to do anything one can do through the Slack API. The full documentation for the Slack API is [here](https://api.slack.com/methods)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authenticate with Slack\n",
    "\n",
    "Replace the line below with the location of your slack api token."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('C:/Users/willk/OneDrive/Desktop/slack_api_python.txt', 'r') as f:\n",
    "    slack_api_token = f.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we authenticate with slack by instantiating a `Slacker` object. We'll test this by checking in our connection is `successful`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully connected to Data Analysis.\n"
     ]
    }
   ],
   "source": [
    "# Connect to Slack\n",
    "slack = Slacker(slack_api_token)\n",
    "if slack.api.test().successful:\n",
    "    print(\n",
    "        f\"Successfully connected to {slack.team.info().body['team']['name']}.\")\n",
    "else:\n",
    "    print('Try Again!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interacting with Slack\n",
    "\n",
    "Anything (okay, nearly anything) you can do in Slack you can do through the Python API. First, let's just do some basic info retrieval. \n",
    "\n",
    "## Retrieving Information\n",
    "\n",
    "Each time you make a request for information, you get back `response` object. The `body` object of the response holds the details of the request (if successful)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ok': True,\n",
       " 'team': {'id': 'TCD256PSM',\n",
       "  'name': 'Data Analysis',\n",
       "  'domain': 'python-data',\n",
       "  'email_domain': 'case.edu',\n",
       "  'icon': {'image_34': 'https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_34.png',\n",
       "   'image_44': 'https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_44.png',\n",
       "   'image_68': 'https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_68.png',\n",
       "   'image_88': 'https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_88.png',\n",
       "   'image_102': 'https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_102.png',\n",
       "   'image_132': 'https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_132.png',\n",
       "   'image_230': 'https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_230.png',\n",
       "   'image_original': 'https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_original.png'}}}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = slack.team.info()\n",
    "r.body"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_230.png\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Image\n",
    "\n",
    "Image(url='https://avatars.slack-edge.com/2018-11-30/492734348690_802c4805ab4a0d29383b_230.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What Can We Do?\n",
    "\n",
    "This is our workspace's icon! (Okay that's not very exciting.) To see everything we can do with Slacker, let's take a look at the methods and attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['get',\n",
       " 'get_presence',\n",
       " 'get_user_id',\n",
       " 'identity',\n",
       " 'info',\n",
       " 'list',\n",
       " 'post',\n",
       " 'set_active',\n",
       " 'set_presence']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "['archive',\n",
       " 'create',\n",
       " 'get',\n",
       " 'get_channel_id',\n",
       " 'history',\n",
       " 'info',\n",
       " 'invite',\n",
       " 'join',\n",
       " 'kick',\n",
       " 'leave',\n",
       " 'list',\n",
       " 'mark',\n",
       " 'post',\n",
       " 'rename',\n",
       " 'replies',\n",
       " 'set_purpose',\n",
       " 'set_topic',\n",
       " 'unarchive']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from utils import get_data_and_model, get_options\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore', category=FutureWarning)\n",
    "\n",
    "command_dict = get_options(slack)\n",
    "command_dict['functions']['users']\n",
    "command_dict['functions']['channels']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can look at all the channels by iterating through them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Channel general Purpose: This channel is for workspace-wide communication and announcements. All members are in this channel.\n",
      "\n",
      "Channel python_content Purpose: \n",
      "\n",
      "Channel python_content_new Purpose: \n",
      "\n",
      "Channel python_content_new2 Purpose: Data Science and Python\n",
      "\n",
      "Channel python_content2 Purpose: Working with Python and Slack\n",
      "\n",
      "Channel python_content3 Purpose: Sharing great python related resources\n",
      "\n",
      "Channel random Purpose: Deep philosophical discussions around the water cooler.\n",
      "\n",
      "Channel slack_interaction Purpose: To interact with Slack from Python\n",
      "\n",
      "Channel testing_exp Purpose: Experimental testing of interactions.\n",
      "\n",
      "Channel training_report Purpose: Report progress while training machine learning models\n",
      "\n"
     ]
    }
   ],
   "source": [
    "channels = slack.channels.list().body\n",
    "\n",
    "# Iterate through channels\n",
    "for channel in channels['channels']:\n",
    "    print(\n",
    "        f'Channel {channel[\"name\"]} Purpose: {channel[\"purpose\"][\"value\"]}\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's the history of the `general` channel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ok': True,\n",
       " 'messages': [{'client_msg_id': 'cd129804-bf06-473c-acb7-52458abcc167',\n",
       "   'type': 'message',\n",
       "   'text': \"It's a little lonely in here. <@USLACKBOT> are you out there?\",\n",
       "   'user': 'UCEKVNHPH',\n",
       "   'ts': '1543624786.001400'},\n",
       "  {'client_msg_id': '65a363a4-265a-4c1d-bb21-914fea4af7fd',\n",
       "   'type': 'message',\n",
       "   'text': \"And please don't answer 42!\",\n",
       "   'user': 'UCEKVNHPH',\n",
       "   'ts': '1543624766.000900'},\n",
       "  {'client_msg_id': '6ace906a-76dd-4482-8001-de3cc8cabcdf',\n",
       "   'type': 'message',\n",
       "   'text': 'What is the meaning of life?',\n",
       "   'user': 'UCEKVNHPH',\n",
       "   'ts': '1543624759.000600'},\n",
       "  {'client_msg_id': 'b43989fc-4e86-4f7d-9ac7-295c0a34597b',\n",
       "   'type': 'message',\n",
       "   'text': 'Anyone going to that Python meetup tonight?',\n",
       "   'user': 'UCEKVNHPH',\n",
       "   'ts': '1543616388.000300'},\n",
       "  {'user': 'UCEKVNHPH',\n",
       "   'type': 'message',\n",
       "   'subtype': 'channel_join',\n",
       "   'ts': '1534988091.000200',\n",
       "   'text': '<@UCEKVNHPH> has joined the channel'}],\n",
       " 'has_more': False}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "slack.channels.history(channel='CCCT28F08').body"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also see all of the users along with their profile photo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "User: wjk68, Real Name: Will Koehrsen, Time Zone: Eastern Standard Time.\n",
      "Current Status: \n"
     ]
    },
    {
     "data": {
      "image/jpeg": "[encoded data, 17316 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "User: slackbot, Real Name: slackbot, Time Zone: Pacific Standard Time.\n",
      "Current Status: \n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 30388 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "users = slack.users.list().body\n",
    "\n",
    "for user in users['members']:\n",
    "    # Print some information\n",
    "    print(\n",
    "        f'\\nUser: {user[\"name\"]}, Real Name: {user[\"real_name\"]}, Time Zone: {user[\"tz_label\"]}.')\n",
    "    print(f'Current Status: {user[\"profile\"][\"status_text\"]}')\n",
    "    # Get image data and show\n",
    "    Image(user['profile']['image_192'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IDs\n",
    "\n",
    "When we interact with a `channel` or `user`, it's generally best to refer to it by the `id`. Therefore, it can be helpful to set up a mapping of channels and users to their ids."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'wjk68': 'UCEKVNHPH', 'slackbot': 'USLACKBOT'}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "channel_dict = {}\n",
    "\n",
    "for channel in channels['channels']:\n",
    "    channel_dict[channel['name']] = channel['id']\n",
    "\n",
    "user_dict = {}\n",
    "\n",
    "for user in users['members']:\n",
    "    user_dict[user['name']] = user['id']\n",
    "\n",
    "user_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Altering Attributes\n",
    "\n",
    "It's pretty lonely in this Slack workspace! Now, we can alter any attributes of users of channels with the ids For example, we can set the purpose of the `python_content` channel as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Channel python_content2 Purpose: Learning how to use Python for Slack interaction\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Set the purpose of the channel\n",
    "r = slack.channels.set_purpose(channel_dict['python_content2'],\n",
    "                               purpose='Learning how to use Python for Slack interaction')\n",
    "\n",
    "channels = slack.channels.list().body\n",
    "\n",
    "# Iterate through channels\n",
    "for channel in channels['channels']:\n",
    "    if channel['name'] == 'python_content2':\n",
    "        print(\n",
    "            f'Channel {channel[\"name\"]} Purpose: {channel[\"purpose\"][\"value\"]}\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's change a few of my attributes. First we can see what I'm up to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Pretty Regular Guy'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'boring'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'Running'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "':runner::skin-tone-5:'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "me = users['members'][0]\n",
    "me['profile']['title']\n",
    "me['profile']['display_name']\n",
    "me['profile']['status_text']\n",
    "me['profile']['status_emoji']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay so not that exciting. Let's change a few things."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "for new_value, name in zip(['Data Scientist and Writer', 'Willk', 'data-sciencing', ':male-technologist:'],\n",
    "                           ['title', 'display_name', 'status_text', 'status_emoji']):\n",
    "\n",
    "    r = slack.users.profile.set(user=user_dict['wjk68'],\n",
    "                                name=name, value=new_value)\n",
    "    if not r:\n",
    "        print(r.error)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Data Scientist and Writer'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "'Willk'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "':male-technologist:'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "users = slack.users.list().body\n",
    "me = users['members'][0]\n",
    "me['profile']['title']\n",
    "me['profile']['display_name']\n",
    "me['profile']['status_text']\n",
    "me['profile']['status_emoji']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The All-Important Emoji\n",
    "\n",
    "Emoji's are set using the `:name:` emoji code notation. The easiest way to figure out which one to use is go to Slack, bring up the emoji's hover over the one you want, and find the name.\n",
    "\n",
    "![](images/emoji_choose.PNG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "🏃\n"
     ]
    }
   ],
   "source": [
    "import emoji\n",
    "\n",
    "\n",
    "def print_emoji(emoji_name):\n",
    "    print(emoji.emojize(emoji_name, use_aliases=True))\n",
    "\n",
    "\n",
    "print_emoji(':runner:')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "💻\n"
     ]
    }
   ],
   "source": [
    "print_emoji(':computer:')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an image of all the available emoji at once. You can find all emoji codes at https://www.webpagefx.com/tools/emoji-cheat-sheet/."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://a.slack-edge.com/c00d19/img/emoji_2017_12_06/sheet_google_64_indexed_256.png\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='https://a.slack-edge.com/c00d19/img/emoji_2017_12_06/sheet_google_64_indexed_256.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Searching\n",
    "\n",
    "You can search through any messages or files in your team's workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_ts(ts):\n",
    "    return pd.to_datetime(ts, unit='s').date()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time: 2018-11-30, Text: Python is better than MatLab! :+1::skin-tone-5:\n",
      "Time: 2018-11-30, Text: Anyone going to that Python meetup tonight?\n"
     ]
    }
   ],
   "source": [
    "matches = slack.search.messages(query='Python').body['messages']['matches']\n",
    "for match in matches:\n",
    "    print(f\"Time: {convert_ts(match['ts'])}, Text: {match['text']}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python is fun 👍\n"
     ]
    }
   ],
   "source": [
    "print(emoji.emojize(\"Python is fun :+1:\", use_aliases=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time: 2018-12-01, Title: Iris Seaborn Plot\n",
      "Time: 2018-11-30, Title: Iris Seaborn Plot\n",
      "Time: 2018-11-30, Title: first plot\n",
      "Time: 2018-11-30, Title: Iris Seaborn Plot\n",
      "Time: 2018-11-30, Title: Iris Seaborn Plot\n",
      "Time: 2018-11-30, Title: Iris Seaborn Plot\n",
      "Time: 2018-08-23, Title: first plot\n",
      "Time: 2018-08-23, Title: first plot\n"
     ]
    }
   ],
   "source": [
    "matches = slack.search.files(query='plot').body['files']['matches']\n",
    "for match in matches:\n",
    "    print(f\"Time: {convert_ts(match['timestamp'])}, Title: {match['title']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Do Not Disturb\n",
    "\n",
    "You can even find the do not disturb settings for everyone on your team."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ok': True,\n",
       " 'users': {'UCEKVNHPH': {'dnd_enabled': True,\n",
       "   'next_dnd_start_ts': 1543633200,\n",
       "   'next_dnd_end_ts': 1543669200}}}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "slack.dnd.team_info().body"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to tell everyone to leave you alone, go ahead and do so! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ok': True,\n",
       " 'snooze_enabled': True,\n",
       " 'snooze_endtime': 1543640381,\n",
       " 'snooze_remaining': 7200}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "slack.dnd.set_snooze(num_minutes=120).body"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And then we can end that snooze becuase there's more to do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ok': True,\n",
       " 'dnd_enabled': True,\n",
       " 'next_dnd_start_ts': 1543633200,\n",
       " 'next_dnd_end_ts': 1543669200,\n",
       " 'snooze_enabled': False}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "slack.dnd.end_snooze().body"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a New Channel\n",
    "\n",
    "Let's make a brand new channel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<slacker.Response at 0x23e99aba860>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<slacker.Response at 0x23ea0a3ada0>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "{'user': 'UCEKVNHPH',\n",
       " 'type': 'message',\n",
       " 'subtype': 'channel_purpose',\n",
       " 'ts': '1543633250.000300',\n",
       " 'text': '<@UCEKVNHPH> set the channel purpose: Data Science and Python',\n",
       " 'purpose': 'Data Science and Python'}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_channel_name = 'python_content_update'\n",
    "\n",
    "# Create new channel\n",
    "r = slack.channels.create(new_channel_name)\n",
    "new_id = slack.channels.get_channel_id(new_channel_name)\n",
    "\n",
    "# Set channel purpose\n",
    "r = slack.channels.set_purpose(new_id, 'Data Science and Python')\n",
    "r = slack.channels.info(new_id)\n",
    "r.body['channel']['latest']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional Options\n",
    "\n",
    "Nearly anything you want to do on Slack you can do through this API! Feel free to check out all of the options (which you can find in the `command_dict`). The [Slack API reference](https://api.slack.com/methods) is helpful for formulating commands."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Posting Messages\n",
    "\n",
    "Now let's get to the heart of why we want to interact with Slack: posting new messages. It's relatively simple to post a message to any channel. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = slack.chat.post_message(channel='python_content',\n",
    "                            text='Have a great day!',\n",
    "                            username='Python Test',\n",
    "                            icon_url='http://devarea.com/wp-content/uploads/2017/11/python-300x300.png')\n",
    "r.successful"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/posted_message.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we set the `username` then the messages are posted as bots. We still get to choose the icon though."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Have a great day!'"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "slack.channels.history(\n",
    "    channel=channel_dict['python_content']).body['messages'][0]['text']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Links in Messages\n",
    "\n",
    "Links are automatically expanded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = slack.chat.post_message(channel='python_content',\n",
    "                            text=\"Here's an uplifting story: https://www.forbes.com/sites/stevedenning/2017/11/30/why-the-world-is-getting-better-why-hardly-anyone-knows-it/#181c93b27826\",\n",
    "                            unfurl_links=True,\n",
    "                            username='Uplift',\n",
    "                            icon_emoji=':point_up::skin-tone-5:')\n",
    "r.successful"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/unfurl.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mentioning Users\n",
    "\n",
    "You can mention specific users with `<@user_id>`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = slack.chat.post_message(channel='python_content',\n",
    "                            text='<@UCEKVNHPH> Have you read any good books lately?',\n",
    "                            link_names=True,\n",
    "                            username='Query',\n",
    "                            icon_emoji=':green_book:')\n",
    "r.successful"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/query.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also post as ourselves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = slack.chat.post_message(channel='python_content',\n",
    "                            text='Anyone up for a trip to the store?',\n",
    "                            as_user='willk')\n",
    "r.successful"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/my_posted_message.PNG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Anyone up for a trip to the store?'"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "slack.channels.history(\n",
    "    channel=channel_dict['python_content']).body['messages'][0]['text']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course you can also notify everyone! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = slack.chat.post_message(channel='python_content',\n",
    "                            text='<!everyone> *This is not a test!*',\n",
    "                            username='Alert',\n",
    "                            icon_emoji=':female-firefighter:')\n",
    "r.successful"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/Alert.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complex Messages\n",
    "\n",
    "Messages can be as complex as you want using attachments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = slack.chat.post_message(channel='random',\n",
    "                            username='Fancy Message',\n",
    "                            icon_emoji=':earth_africa:',\n",
    "                            attachments=[\n",
    "                                {\n",
    "                                    \"fallback\": \"Required plain-text summary of the attachment.\",\n",
    "                                    \"color\": \"#2eb886\",\n",
    "                                    \"pretext\": \"Optional text that appears above the attachment block\",\n",
    "                                    \"author_name\": \"Fancy Message\",\n",
    "                                    \"title\": \"Slack API Documentation\",\n",
    "                                    \"title_link\": \"https://api.slack.com/\",\n",
    "                                    \"text\": \"Optional text that appears within the attachment\",\n",
    "                                    \"fields\": [\n",
    "                                        {\n",
    "                                            \"title\": \"Priority\",\n",
    "                                            \"value\": \"High\",\n",
    "                                            \"short\": False\n",
    "                                        }\n",
    "                                    ],\n",
    "                                    \"image_url\": \"http://my-website.com/path/to/image.jpg\",\n",
    "                                    \"thumb_url\": \"http://example.com/path/to/thumb.png\",\n",
    "                                    \"footer\": \"Slack API\",\n",
    "                                    \"footer_icon\": \"https://platform.slack-edge.com/img/default_application_icon.png\",\n",
    "                                    \"ts\": 123456789\n",
    "                                }\n",
    "                            ]\n",
    "                            )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/complex.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can even do crazy things like posting clickable games! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = slack.chat.post_message(channel='random', text='Play a game!',\n",
    "                            username='Game Overseer',\n",
    "                            icon_emoji=':black_joker:',\n",
    "                            attachments=[\n",
    "                                {\n",
    "                                    \"text\": \"Choose a game to play\",\n",
    "                                    \"fallback\": \"You are unable to choose a game\",\n",
    "                                    \"callback_id\": \"wopr_game\",\n",
    "                                    \"color\": \"#3AA3E3\",\n",
    "                                    \"attachment_type\": \"default\",\n",
    "                                    \"actions\": [\n",
    "                                        {\n",
    "                                            \"name\": \"game\",\n",
    "                                            \"text\": \"Chess\",\n",
    "                                            \"type\": \"button\",\n",
    "                                            \"value\": \"chess\",\n",
    "                                            \"confirm\": {\n",
    "                                                \"title\": \"Chess is a great game for the mind\",\n",
    "                                                \"text\": \"Are you up to the challenge?\",\n",
    "                                                \"ok_text\": \"Yes\",\n",
    "                                                \"dismiss_text\": \"No\"\n",
    "                                            }\n",
    "                                        },\n",
    "                                        {\n",
    "                                            \"name\": \"game\",\n",
    "                                            \"text\": \"Falken's Maze\",\n",
    "                                            \"type\": \"button\",\n",
    "                                            \"value\": \"maze\",\n",
    "                                            \"confirm\": {\n",
    "                                                \"title\": \"Mazes aren't that interesting!\",\n",
    "                                                \"text\": \"How about the other options?\",\n",
    "                                                \"ok_text\": \"Yes\",\n",
    "                                                \"dismiss_text\": \"No\"\n",
    "                                            }\n",
    "                                        },\n",
    "                                        {\n",
    "                                            \"name\": \"game\",\n",
    "                                            \"text\": \"Thermonuclear War\",\n",
    "                                            \"style\": \"danger\",\n",
    "                                            \"type\": \"button\",\n",
    "                                            \"value\": \"war\",\n",
    "                                            \"confirm\": {\n",
    "                                                \"title\": \"Are you sure?\",\n",
    "                                                \"text\": \"Wouldn't you prefer a good game of chess?\",\n",
    "                                                \"ok_text\": \"Yes\",\n",
    "                                                \"dismiss_text\": \"No\"\n",
    "                                            }\n",
    "                                        }\n",
    "                                    ]\n",
    "                                }\n",
    "                            ]\n",
    "                            )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/game_button.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Uploading Files\n",
    "\n",
    "One of my favorite parts of Slack is uploading files. This is done using a local file stored on your machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = slack.files.upload(file_='images/volcano_crater.jpg',\n",
    "                       channels=['random'], title='Volcano Crater',\n",
    "                       initial_comment='This would make a great display background')\n",
    "r.successful"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's the image that we posted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://files.slack.com/files-tmb/TCD256PSM-FEGP4J4HH-5254b39955/volcano_crater_480.jpg'"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img_url = r.body['file']['thumb_480']\n",
    "img_url"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/jpeg": "[encoded data, 302092 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image('images/volcano_crater.jpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Posting Plots\n",
    "\n",
    "Once we know how to upload files, we can start posting results of our analyses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "code_folding": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 15656 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore', category=FutureWarning)\n",
    "\n",
    "# library & dataset\n",
    "df = sns.load_dataset('iris')\n",
    "\n",
    "# Basic 2D density plot\n",
    "sns.set_style(\"white\")\n",
    "\n",
    "# Some features are characteristic of 2D: color palette and wether or not color the lowest range\n",
    "sns.kdeplot(df.sepal_width, df.sepal_length,\n",
    "            cmap=\"Blues\", shade=True, shade_lowest=True, )\n",
    "plt.title('Good Ole Iris Data Set')\n",
    "\n",
    "# Save last figure\n",
    "plt.savefig('iris_plot.png', dpi=500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = slack.files.upload(file_='iris_plot.png',\n",
    "                       channels=['slack_interaction', 'python_content3'],\n",
    "                       title='Iris Seaborn Plot',\n",
    "                       initial_comment=\"I've seen this data set way too many times. Seaborn makes some nice plots though.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/plot_posted.PNG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 30716 characters]\n",
      "text/plain": [
       "<Figure size 792x648 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from string import ascii_letters\n",
    "\n",
    "d = pd.DataFrame(data=np.random.normal(size=(100, 26)),\n",
    "                 columns=list(ascii_letters[26:]))\n",
    "\n",
    "# Compute the correlation matrix\n",
    "corr = d.corr()\n",
    "\n",
    "# Generate a mask for the upper triangle\n",
    "mask = np.zeros_like(corr, dtype=np.bool)\n",
    "mask[np.triu_indices_from(mask)] = True\n",
    "\n",
    "# Set up the matplotlib figure\n",
    "f, ax = plt.subplots(figsize=(11, 9))\n",
    "\n",
    "# Generate a custom diverging colormap\n",
    "cmap = sns.diverging_palette(220, 10, as_cmap=True)\n",
    "\n",
    "# Draw the heatmap with the mask and correct aspect ratio\n",
    "sns.heatmap(corr, mask=mask, cmap=cmap, vmax=.3, center=0,\n",
    "            square=True, linewidths=.5, cbar_kws={\"shrink\": .5})\n",
    "\n",
    "plt.savefig('heatmap_ex.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = slack.files.upload(file_='heatmap_ex.png',\n",
    "                       channels=['slack_interaction', 'python_content3'],\n",
    "                       title='Correlation Heatmap',\n",
    "                       initial_comment=\"The correlations are meaningless becuase they are random. Nice plot though.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Monitor Training\n",
    "\n",
    "Let's get serious and use slack for some real work. Here we'll monitor the progess of a convolutional neural network as it is training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we'll create a new channel if it does not yet exist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'user': 'UCEKVNHPH',\n",
       " 'type': 'message',\n",
       " 'subtype': 'channel_topic',\n",
       " 'ts': '1543633470.009500',\n",
       " 'text': '<@UCEKVNHPH> set the channel topic: Progress Monitoring',\n",
       " 'topic': 'Progress Monitoring'}"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "channel_list = [c['name'] for c in slack.channels.list().body['channels']]\n",
    "\n",
    "reporting_channel = 'training_report'\n",
    "\n",
    "if reporting_channel not in channel_list:\n",
    "    slack.channels.create('training_report')\n",
    "\n",
    "cid = slack.channels.get_channel_id('training_report')\n",
    "r = slack.channels.set_purpose(\n",
    "    cid, 'Report progress while training machine learning models')\n",
    "r = slack.channels.set_topic(cid, 'Progress Monitoring')\n",
    "r = slack.channels.info(cid).body\n",
    "r['channel']['latest']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model and Data\n",
    "\n",
    "Now we'll import some data and a model. This is just a basic convolutional neural network that will be trained to identify numbers (MNIST). The script is from https://github.com/keras-team/keras/blob/master/examples/mnist_cnn.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28, 1)\n",
      "60000 train samples\n",
      "10000 test samples\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_1 (Conv2D)            (None, 26, 26, 32)        320       \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 24, 24, 64)        18496     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 12, 12, 64)        0         \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 12, 12, 64)        0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 9216)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 128)               1179776   \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 1,199,882\n",
      "Trainable params: 1,199,882\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from utils import get_data_and_model\n",
    "\n",
    "x_train, x_test, y_train, y_test, model = get_data_and_model()\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Keras Custom Callback\n",
    "\n",
    "We'll write a quick custom Keras callback that reports the progress while training. Refer to [the documentation](https://keras.io/callbacks/) for specifics. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.callbacks import Callback\n",
    "from datetime import datetime\n",
    "\n",
    "\n",
    "def report_stats(text, channel):\n",
    "    \"\"\"Report training stats\"\"\"\n",
    "    r = slack.chat.post_message(channel=channel, text=text,\n",
    "                                username='Training Report',\n",
    "                                icon_emoji=':clipboard:')\n",
    "\n",
    "    if r.successful:\n",
    "        return True\n",
    "    else:\n",
    "        return r.error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "from timeit import default_timer as timer\n",
    "\n",
    "\n",
    "class SlackUpdate(Callback):\n",
    "    \"\"\"Custom Keras callback that posts to Slack while training a neural network\"\"\"\n",
    "\n",
    "    def __init__(self, channel):\n",
    "        self.channel = channel\n",
    "\n",
    "    def on_train_begin(self, logs={}):\n",
    "        report_stats(text=f'Training started at {datetime.now()}',\n",
    "                     channel=reporting_channel)\n",
    "\n",
    "        self.start_time = timer()\n",
    "        self.train_acc = []\n",
    "        self.valid_acc = []\n",
    "        self.train_loss = []\n",
    "        self.valid_loss = []\n",
    "        self.n_epochs = 0\n",
    "\n",
    "    def on_epoch_end(self, batch, logs={}):\n",
    "\n",
    "        self.train_acc.append(logs.get('acc'))\n",
    "        self.valid_acc.append(logs.get('val_acc'))\n",
    "        self.train_loss.append(logs.get('loss'))\n",
    "        self.valid_loss.append(logs.get('val_loss'))\n",
    "        self.n_epochs += 1\n",
    "\n",
    "        message = f'Epoch: {self.n_epochs} Training Loss: {self.train_loss[-1]:.4f} Validation Loss: {self.valid_loss[-1]:.4f}'\n",
    "\n",
    "        report_stats(message, channel=self.channel)\n",
    "\n",
    "    def on_train_end(self, logs={}):\n",
    "\n",
    "        best_epoch = np.argmin(self.valid_loss)\n",
    "        valid_loss = self.valid_loss[best_epoch]\n",
    "        train_loss = self.train_loss[best_epoch]\n",
    "        train_acc = self.train_acc[best_epoch]\n",
    "        valid_acc = self.valid_acc[best_epoch]\n",
    "\n",
    "        message = f'Trained for {self.n_epochs} epochs. Best epoch was {best_epoch + 1}.'\n",
    "        report_stats(message, channel=self.channel)\n",
    "        message = f'Best validation loss = {valid_loss:.4f} Training Loss = {train_loss:.2f} Validation accuracy = {100*valid_acc:.2f}%'\n",
    "        report_stats(message, channel=self.channel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's instantiate this callback. We'll report to the new channel we created. \n",
    "\n",
    "Then we start training and can monitor the training in the channel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 36000 samples, validate on 24000 samples\n",
      "Epoch 1/12\n",
      "36000/36000 [==============================] - 18s 491us/step - loss: 0.7556 - acc: 0.7602 - val_loss: 0.2250 - val_acc: 0.9309\n",
      "Epoch 2/12\n",
      "36000/36000 [==============================] - 9s 263us/step - loss: 0.2029 - acc: 0.9413 - val_loss: 0.1062 - val_acc: 0.9685\n",
      "Epoch 3/12\n",
      "36000/36000 [==============================] - 10s 267us/step - loss: 0.1291 - acc: 0.9616 - val_loss: 0.1180 - val_acc: 0.9632\n",
      "Epoch 4/12\n",
      "36000/36000 [==============================] - 10s 272us/step - loss: 0.1004 - acc: 0.9697 - val_loss: 0.0744 - val_acc: 0.9775\n",
      "Epoch 5/12\n",
      "36000/36000 [==============================] - 10s 279us/step - loss: 0.0870 - acc: 0.9736 - val_loss: 0.0671 - val_acc: 0.9799\n",
      "Epoch 6/12\n",
      "36000/36000 [==============================] - 10s 269us/step - loss: 0.0728 - acc: 0.9782 - val_loss: 0.0659 - val_acc: 0.9797\n",
      "Epoch 7/12\n",
      "36000/36000 [==============================] - 10s 274us/step - loss: 0.0655 - acc: 0.9804 - val_loss: 0.0548 - val_acc: 0.9835\n",
      "Epoch 8/12\n",
      "36000/36000 [==============================] - 10s 271us/step - loss: 0.0572 - acc: 0.9822 - val_loss: 0.0556 - val_acc: 0.9837\n",
      "Epoch 9/12\n",
      "36000/36000 [==============================] - 10s 268us/step - loss: 0.0509 - acc: 0.9844 - val_loss: 0.0534 - val_acc: 0.9839\n",
      "Epoch 10/12\n",
      "36000/36000 [==============================] - 10s 273us/step - loss: 0.0482 - acc: 0.9849 - val_loss: 0.0473 - val_acc: 0.9859\n",
      "Epoch 11/12\n",
      "36000/36000 [==============================] - 10s 284us/step - loss: 0.0420 - acc: 0.9878 - val_loss: 0.0500 - val_acc: 0.9849\n",
      "Epoch 12/12\n",
      "36000/36000 [==============================] - 10s 267us/step - loss: 0.0404 - acc: 0.9878 - val_loss: 0.0486 - val_acc: 0.9863\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "updater = SlackUpdate(channel=reporting_channel)\n",
    "\n",
    "\n",
    "# Fit the model for 10 epochs\n",
    "history = model.fit(x_train, y_train, epochs=12, batch_size=512,\n",
    "                    callbacks=[updater], validation_split=0.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have a thorough report of what happened during training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/training_report.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot Results\n",
    "\n",
    "Next we can create a plot of the training curves and send them to the reporting channel. This could also be included in the custom callback `on_train_end` method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we'll plot the figure and save it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 69520 characters]\n",
      "text/plain": [
       "<Figure size 1296x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from utils import plot_history\n",
    "\n",
    "plot_history(history.history)\n",
    "plt.savefig('training_curves.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we upload the file along with some helpful comments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Best loss of 0.0473 at epoch 10.'"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find minimum loss\n",
    "min_loss = min(history.history['val_loss'])\n",
    "best_epoch = np.argmin(history.history['val_loss']) + 1\n",
    "\n",
    "# Upload file\n",
    "comment = f\"Best loss of {min_loss:.4f} at epoch {best_epoch}.\"\n",
    "r = slack.files.upload(file_='training_curves.png', title=\"Training Curves\", channels=[reporting_channel],\n",
    "                       initial_comment=comment)\n",
    "r = slack.channels.info(cid).body\n",
    "r['channel']['latest']['text']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/training_plot.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload Predictions\n",
    "\n",
    "We'll were at it, we might as well upload some predictions on test images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import MatplotlibDeprecationWarning\n",
    "warnings.filterwarnings('ignore', category=MatplotlibDeprecationWarning)\n",
    "\n",
    "\n",
    "def plot_image(image, ax=None):\n",
    "    if ax is not None:\n",
    "        ax.imshow(image.reshape((28, 28)), cmap='Greys')\n",
    "    else:\n",
    "        plt.imshow(image.reshape((28, 28)), cmap='Greys')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 7900 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_image(x_test[10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 1., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_predictions(n=4):\n",
    "    \"\"\"Plot test image and predictions\"\"\"\n",
    "\n",
    "    # Get random images to plot\n",
    "    to_plot = np.random.choice(list(range(x_test.shape[0])),\n",
    "                               size=n, replace=False)\n",
    "    correct = []\n",
    "    # Make predictions and plot each image\n",
    "    for i in to_plot:\n",
    "        image = x_test[i]\n",
    "        probs = model.predict_proba(image.reshape((1, 28, 28, 1)))[0]\n",
    "        pred = pd.DataFrame({'prob': probs})\n",
    "        fig, axs = plt.subplots(1, 2, figsize=(16, 6))\n",
    "        plot_image(image, axs[0])\n",
    "\n",
    "        pred['prob'].plot.bar(ax=axs[1])\n",
    "        axs[1].set_xlabel('Class')\n",
    "        axs[1].set_ylabel('Probability')\n",
    "        axs[1].set_title('Predictions')\n",
    "        plt.savefig(f'images/test-{i}-predictions.png')\n",
    "        plt.show()\n",
    "\n",
    "        correct.append(np.argmax(probs) == np.argmax(y_test[i]))\n",
    "\n",
    "    return to_plot, correct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 27332 characters]\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 27144 characters]\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 26820 characters]\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 27392 characters]\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "to_plot, c = plot_predictions()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "def post_predictions(channel, n=4):\n",
    "    \"\"\"Post Keras preditions to Slack\"\"\"\n",
    "\n",
    "    # Make predictions\n",
    "    plot_indexes, correct = plot_predictions(n=n)\n",
    "\n",
    "    # Iterate through images and correct indicators\n",
    "    for i, r in zip(plot_indexes, correct):\n",
    "        filename = f'images/test-{i}-predictions.png'\n",
    "        # Upload the files\n",
    "        r = slack.files.upload(file_=filename,\n",
    "                               title=\"Predictions\",\n",
    "                               channels=[channel],\n",
    "                               initial_comment='Correct' if r else 'Incorrect')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 27532 characters]\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 27212 characters]\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 26636 characters]\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 27484 characters]\n",
      "text/plain": [
       "<Figure size 1152x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "post_predictions(reporting_channel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is what the predictions look like uploaded to Slack. \n",
    "\n",
    "![](images/prediction_plots.PNG)\n",
    "\n",
    "Looks like this model does pretty well. If you set up a model to train overnight, it would be nice to see these results in the morning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusions\n",
    "\n",
    "While apps such as Slack can be used for fun, they also can improve productivity when used correctly. Once we realize the nearly all of these apps have Python apis, we can start to figure out how to use them programmatically for the greatest efficiency.\n",
    "\n",
    "In this notebook, we saw how to interact with Slack using the `Slacker` library in Python. The library allows us to do anything we can do through the Slack API which gives us control over almost every aspect of our workspace. \n",
    "\n",
    "We can do things such as:\n",
    "\n",
    "* Retrieve channel, message, and user information\n",
    "* Alter and manage settings\n",
    "* Post messages including complex formatted apps\n",
    "* Upload files such as Python plots\n",
    "* Track the progress of a machine learning training run\n",
    "* Post results of a deep learning exercise \n",
    "\n",
    "We have only covered the surface level of what we can do programmatically. Although this particular use case may not be compelling to you, the overarching idea that we can use Python to carry out many routine tasks and improve our efficiency is powerful. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
