Merge pull request #1 from aymericdamien/dev

 Change layout + Add Notebooks

LICENSE

@@ -0,0 +1,30 @@
+The MIT License (MIT)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+All contributions by Aymeric Damien:
+Copyright (c) 2015, Aymeric Damien.
+All rights reserved.
+
+All other contributions:
+Copyright (c) 2015, the respective contributors.
+All rights reserved.
+
+Each contributor holds copyright over their respective contributions.
+The project versioning (Git) records all such contribution source information.

README.md

@@ -1,25 +1,25 @@
 # TensorFlow Examples
-Basic code examples for some machine learning algorithms, using TensorFlow library.
+Code examples for some popular machine learning algorithms, using TensorFlow library. This tutorial is designed to easily dive into TensorFlow, through examples. It includes both notebook and code with explanations.
 
 ## Tutorial index
 
 #### 1 - Introduction
-- Hello World ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/helloworld.py))
-- Basic Operations ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/basic_operations.py))
+- Hello World ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/1%20-%20Introduction/helloworld.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/1%20-%20Introduction/helloworld.py))
+- Basic Operations ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/1%20-%20Introduction/basic_operations.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/1%20-%20Introduction/basic_operations.py))
 
 #### 2 - Basic Classifiers
-- Nearest Neighbor ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/nearest_neighbor.py))
-- Linear Regression ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/linear_regression.py))
-- Logistic Regression ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/logistic_regression.py))
+- Nearest Neighbor ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/2%20-%20Basic%20Classifiers/nearest_neighbor.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/2%20-%20Basic%20Classifiers/nearest_neighbor.py))
+- Linear Regression ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/2%20-%20Basic%20Classifiers/linear_regression.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/2%20-%20Basic%20Classifiers/linear_regression.py))
+- Logistic Regression ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/2%20-%20Basic%20Classifiers/logistic_regression.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/2%20-%20Basic%20Classifiers/logistic_regression.py))
 
 #### 3 - Neural Networks
-- Multilayer Perceptron ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/multilayer_perceptron.py))
-- Convolutional Neural Network ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/convolutional_network.py))
-- AlexNet ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/alexnet.py))
-- Reccurent Network ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/recurrent_network.py))
+- Multilayer Perceptron ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/3%20-%20Neural%20Networks/multilayer_perceptron.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3%20-%20Neural%20Networks/multilayer_perceptron.py))
+- Convolutional Neural Network ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/3%20-%20Neural%20Networks/convolutional_network.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3%20-%20Neural%20Networks/convolutional_network.py))
+- AlexNet ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/3%20-%20Neural%20Networks/alexnet.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3%20-%20Neural%20Networks/alexnet.py))
+- Reccurent Network ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3%20-%20Neural%20Networks/recurrent_network.py))
 
 ### 4 - Multi GPU
-- Basic Operations on multi-GPU ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/multigpu_basics.py))
+- Basic Operations on multi-GPU ([notebook](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/4%20-%20Multi%20GPU/multigpu_basics.ipynb)) ([code](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/4%20-%20Multi%20GPU/multigpu_basics.py))
 
 ## Dependencies
 ```
@@ -28,5 +28,10 @@ numpy
 matplotlib
 cuda (to run examples on GPU)
 ```
+For more details about TensorFlow installation, you can check [Setup_TensorFlow.md](https://github.com/aymericdamien/TensorFlow-Examples/blob/master/Setup_TensorFlow.md)
+
+## Dataset
+Some examples require MNIST dataset for training and testing. Don't worry, this dataset will automatically be downloaded when running examples (with input_data.py).
+MNIST is a database of handwritten digits, with 60,000 examples for training and 10,000 examples for testing. (Website: [http://yann.lecun.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/))
 
 _Other tutorials are coming soon..._

Setup_TensorFlow.md

@@ -0,0 +1,486 @@
+_From TensorFlow Official docs_
+
+# Download and Setup <a class="md-anchor" id="AUTOGENERATED-download-and-setup"></a>
+
+You can install TensorFlow using our provided binary packages or from source.
+
+## Binary Installation <a class="md-anchor" id="AUTOGENERATED-binary-installation"></a>
+
+The TensorFlow Python API currently requires Python 2.7: we are
+[working](https://github.com/tensorflow/tensorflow/issues/1) on adding support
+for Python 3.
+
+The simplest way to install TensorFlow is using
+[pip](https://pypi.python.org/pypi/pip) for both Linux and Mac.
+
+If you encounter installation errors, see
+[common problems](#common_install_problems) for some solutions. To simplify
+installation, please consider using our virtualenv-based instructions
+[here](#virtualenv_install).
+
+### Ubuntu/Linux 64-bit <a class="md-anchor" id="AUTOGENERATED-ubuntu-linux-64-bit"></a>
+
+```bash
+# For CPU-only version
+$ pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
+
+# For GPU-enabled version (only install this version if you have the CUDA sdk installed)
+$ pip install https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
+```
+
+### Mac OS X <a class="md-anchor" id="AUTOGENERATED-mac-os-x"></a>
+
+On OS X, we recommend installing [homebrew](http://brew.sh) and `brew install
+python` before proceeding, or installing TensorFlow within [virtualenv](#virtualenv_install).
+
+```bash
+# Only CPU-version is available at the moment.
+$ pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
+```
+
+## Docker-based installation <a class="md-anchor" id="AUTOGENERATED-docker-based-installation"></a>
+
+We also support running TensorFlow via [Docker](http://docker.com/), which lets
+you avoid worrying about setting up dependencies.
+
+First, [install Docker](http://docs.docker.com/engine/installation/). Once
+Docker is up and running, you can start a container with one command:
+
+```bash
+$ docker run -it b.gcr.io/tensorflow/tensorflow
+```
+
+This will start a container with TensorFlow and all its dependencies already
+installed.
+
+### Additional images <a class="md-anchor" id="AUTOGENERATED-additional-images"></a>
+
+The default Docker image above contains just a minimal set of libraries for
+getting up and running with TensorFlow. We also have the following container,
+which you can use in the `docker run` command above:
+
+* `b.gcr.io/tensorflow/tensorflow-full`: Contains a complete TensorFlow source
+  installation, including all utilities needed to build and run TensorFlow. This
+  makes it easy to experiment directly with the source, without needing to
+  install any of the dependencies described above.
+
+## VirtualEnv-based installation <a class="md-anchor" id="virtualenv_install"></a>
+
+We recommend using [virtualenv](https://pypi.python.org/pypi/virtualenv) to
+create an isolated container and install TensorFlow in that container -- it is
+optional but makes verifying installation issues easier.
+
+First, install all required tools:
+
+```bash
+# On Linux:
+$ sudo apt-get install python-pip python-dev python-virtualenv
+
+# On Mac:
+$ sudo easy_install pip  # If pip is not already installed
+$ sudo pip install --upgrade virtualenv
+```
+
+Next, set up a new virtualenv environment.  To set it up in the
+directory `~/tensorflow`, run:
+
+```bash
+$ virtualenv --system-site-packages ~/tensorflow
+$ cd ~/tensorflow
+```
+
+Then activate the virtualenv:
+
+```bash
+$ source bin/activate  # If using bash
+$ source bin/activate.csh  # If using csh
+(tensorflow)$  # Your prompt should change
+```
+
+Inside the virtualenv, install TensorFlow:
+
+```bash
+# For CPU-only linux x86_64 version
+(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
+
+# For GPU-enabled linux x86_64 version
+(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
+
+# For Mac CPU-only version
+(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
+```
+
+Make sure you have downloaded the source code for TensorFlow, and then you can
+then run an example TensorFlow program like:
+
+```bash
+(tensorflow)$ cd tensorflow/models/image/mnist
+(tensorflow)$ python convolutional.py
+
+# When you are done using TensorFlow:
+(tensorflow)$ deactivate  # Deactivate the virtualenv
+
+$  # Your prompt should change back
+```
+
+## Try your first TensorFlow program <a class="md-anchor" id="AUTOGENERATED-try-your-first-tensorflow-program"></a>
+
+### (Optional) Enable GPU Support <a class="md-anchor" id="AUTOGENERATED--optional--enable-gpu-support"></a>
+
+If you installed the GPU-enabled TensorFlow pip binary, you must have the
+correct versions of the CUDA SDK and CUDNN installed on your
+system.  Please see [the CUDA installation instructions](#install_cuda).
+
+You also need to set the `LD_LIBRARY_PATH` and `CUDA_HOME` environment
+variables.  Consider adding the commands below to your `~/.bash_profile`.  These
+assume your CUDA installation is in `/usr/local/cuda`:
+
+```bash
+export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64"
+export CUDA_HOME=/usr/local/cuda
+```
+
+### Run TensorFlow <a class="md-anchor" id="AUTOGENERATED-run-tensorflow"></a>
+
+Open a python terminal:
+
+```bash
+$ python
+
+>>> import tensorflow as tf
+>>> hello = tf.constant('Hello, TensorFlow!')
+>>> sess = tf.Session()
+>>> print sess.run(hello)
+Hello, TensorFlow!
+>>> a = tf.constant(10)
+>>> b = tf.constant(32)
+>>> print sess.run(a+b)
+42
+>>>
+
+```
+
+## Installing from sources <a class="md-anchor" id="source"></a>
+
+### Clone the TensorFlow repository <a class="md-anchor" id="AUTOGENERATED-clone-the-tensorflow-repository"></a>
+
+```bash
+$ git clone --recurse-submodules https://github.com/tensorflow/tensorflow
+```
+
+`--recurse-submodules` is required to fetch the protobuf library that TensorFlow
+depends on.
+
+### Installation for Linux <a class="md-anchor" id="AUTOGENERATED-installation-for-linux"></a>
+
+#### Install Bazel <a class="md-anchor" id="AUTOGENERATED-install-bazel"></a>
+
+
+Follow instructions [here](http://bazel.io/docs/install.html) to install the
+dependencies for Bazel. Then download bazel version 0.1.1 using the
+[installer for your system](https://github.com/bazelbuild/bazel/releases) and
+run the installer as mentioned there:
+
+```bash
+$ chmod +x PATH_TO_INSTALL.SH
+$ ./PATH_TO_INSTALL.SH --user
+```
+
+Remember to replace `PATH_TO_INSTALL.SH` to point to the location where you
+downloaded the installer.
+
+Finally, follow the instructions in that script to place bazel into your binary
+path.
+
+#### Install other dependencies <a class="md-anchor" id="AUTOGENERATED-install-other-dependencies"></a>
+
+```bash
+$ sudo apt-get install python-numpy swig python-dev
+```
+
+#### Optional: Install CUDA (GPUs on Linux) <a class="md-anchor" id="install_cuda"></a>
+
+In order to build or run TensorFlow with GPU support, both Cuda Toolkit 7.0 and
+CUDNN 6.5 V2 from NVIDIA need to be installed.
+
+TensorFlow GPU support requires having a GPU card with NVidia Compute Capability >= 3.5.  Supported cards include but are not limited to:
+
+* NVidia Titan
+* NVidia Titan X
+* NVidia K20
+* NVidia K40
+
+##### Download and install Cuda Toolkit 7.0 <a class="md-anchor" id="AUTOGENERATED-download-and-install-cuda-toolkit-7.0"></a>
+
+https://developer.nvidia.com/cuda-toolkit-70
+
+Install the toolkit into e.g. `/usr/local/cuda`
+
+##### Download and install CUDNN Toolkit 6.5 <a class="md-anchor" id="AUTOGENERATED-download-and-install-cudnn-toolkit-6.5"></a>
+
+https://developer.nvidia.com/rdp/cudnn-archive
+
+Uncompress and copy the cudnn files into the toolkit directory.  Assuming the
+toolkit is installed in `/usr/local/cuda`:
+
+``` bash
+tar xvzf cudnn-6.5-linux-x64-v2.tgz
+sudo cp cudnn-6.5-linux-x64-v2/cudnn.h /usr/local/cuda/include
+sudo cp cudnn-6.5-linux-x64-v2/libcudnn* /usr/local/cuda/lib64
+```
+
+##### Configure TensorFlow's canonical view of Cuda libraries <a class="md-anchor" id="AUTOGENERATED-configure-tensorflow-s-canonical-view-of-cuda-libraries"></a>
+From the root of your source tree, run:
+
+``` bash
+$ ./configure
+Do you wish to build TensorFlow with GPU support? [y/n] y
+GPU support will be enabled for TensorFlow
+
+Please specify the location where CUDA 7.0 toolkit is installed. Refer to
+README.md for more details. [default is: /usr/local/cuda]: /usr/local/cuda
+
+Please specify the location where CUDNN 6.5 V2 library is installed. Refer to
+README.md for more details. [default is: /usr/local/cuda]: /usr/local/cuda
+
+Setting up Cuda include
+Setting up Cuda lib64
+Setting up Cuda bin
+Setting up Cuda nvvm
+Configuration finished
+```
+
+This creates a canonical set of symbolic links to the Cuda libraries on your system.
+Every time you change the Cuda library paths you need to run this step again before
+you invoke the bazel build command.
+
+##### Build your target with GPU support. <a class="md-anchor" id="AUTOGENERATED-build-your-target-with-gpu-support."></a>
+From the root of your source tree, run:
+
+```bash
+$ bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer
+
+$ bazel-bin/tensorflow/cc/tutorials_example_trainer --use_gpu
+# Lots of output. This tutorial iteratively calculates the major eigenvalue of
+# a 2x2 matrix, on GPU. The last few lines look like this.
+000009/000005 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
+000006/000001 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
+000009/000009 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]
+```
+
+Note that "--config=cuda" is needed to enable the GPU support.
+
+##### Enabling Cuda 3.0. <a class="md-anchor" id="AUTOGENERATED-enabling-cuda-3.0."></a>
+TensorFlow officially supports Cuda devices with 3.5 and 5.2 compute
+capabilities. In order to enable earlier Cuda devices such as Grid K520, you
+need to target Cuda 3.0. This can be done through TensorFlow unofficial
+settings with "configure".
+
+```bash
+$ TF_UNOFFICIAL_SETTING=1 ./configure
+
+# Same as the official settings above
+
+WARNING: You are configuring unofficial settings in TensorFlow. Because some
+external libraries are not backward compatible, these settings are largely
+untested and unsupported.
+
+Please specify a list of comma-separated Cuda compute capabilities you want to
+build with. You can find the compute capability of your device at:
+https://developer.nvidia.com/cuda-gpus.
+Please note that each additional compute capability significantly increases
+your build time and binary size. [Default is: "3.5,5.2"]: 3.0
+
+Setting up Cuda include
+Setting up Cuda lib64
+Setting up Cuda bin
+Setting up Cuda nvvm
+Configuration finished
+```
+
+##### Known issues <a class="md-anchor" id="AUTOGENERATED-known-issues"></a>
+
+* Although it is possible to build both Cuda and non-Cuda configs under the same
+source tree, we recommend to run "bazel clean" when switching between these two
+configs in the same source tree.
+
+* You have to run configure before running bazel build. Otherwise, the build
+will fail with a clear error message. In the future, we might consider making
+this more conveninent by including the configure step in our build process,
+given necessary bazel new feature support.
+
+### Installation for Mac OS X <a class="md-anchor" id="AUTOGENERATED-installation-for-mac-os-x"></a>
+
+Mac needs the same set of dependencies as Linux, however their installing those
+dependencies is different. Here is a set of useful links to help with installing
+the dependencies on Mac OS X :
+
+#### Bazel <a class="md-anchor" id="AUTOGENERATED-bazel"></a>
+
+Look for installation instructions for Mac OS X on
+[this](http://bazel.io/docs/install.html) page.
+
+#### SWIG <a class="md-anchor" id="AUTOGENERATED-swig"></a>
+
+[Mac OS X installation](http://www.swig.org/Doc3.0/Preface.html#Preface_osx_installation).
+
+Notes : You need to install
+[PCRE](ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/) and *NOT* PCRE2.
+
+#### Numpy <a class="md-anchor" id="AUTOGENERATED-numpy"></a>
+
+Follow installation instructions [here](http://docs.scipy.org/doc/numpy/user/install.html).
+
+
+### Create the pip package and install <a class="md-anchor" id="create-pip"></a>
+
+```bash
+$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
+
+# To build with GPU support:
+$ bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
+
+$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
+
+# The name of the .whl file will depend on your platform.
+$ pip install /tmp/tensorflow_pkg/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
+```
+
+## Train your first TensorFlow neural net model <a class="md-anchor" id="AUTOGENERATED-train-your-first-tensorflow-neural-net-model"></a>
+
+Starting from the root of your source tree, run:
+
+```python
+$ cd tensorflow/models/image/mnist
+$ python convolutional.py
+Succesfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
+Succesfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
+Succesfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
+Succesfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
+Extracting data/train-images-idx3-ubyte.gz
+Extracting data/train-labels-idx1-ubyte.gz
+Extracting data/t10k-images-idx3-ubyte.gz
+Extracting data/t10k-labels-idx1-ubyte.gz
+Initialized!
+Epoch 0.00
+Minibatch loss: 12.054, learning rate: 0.010000
+Minibatch error: 90.6%
+Validation error: 84.6%
+Epoch 0.12
+Minibatch loss: 3.285, learning rate: 0.010000
+Minibatch error: 6.2%
+Validation error: 7.0%
+...
+...
+```
+
+## Common Problems <a class="md-anchor" id="common_install_problems"></a>
+
+### GPU-related issues <a class="md-anchor" id="AUTOGENERATED-gpu-related-issues"></a>
+
+If you encounter the following when trying to run a TensorFlow program:
+
+```python
+ImportError: libcudart.so.7.0: cannot open shared object file: No such file or directory
+```
+
+Make sure you followed the the GPU installation [instructions](#install_cuda).
+
+### Pip installation issues <a class="md-anchor" id="AUTOGENERATED-pip-installation-issues"></a>
+
+#### Can't find setup.py <a class="md-anchor" id="AUTOGENERATED-can-t-find-setup.py"></a>
+
+If, during `pip install`, you encounter an error like:
+
+```bash
+...
+IOError: [Errno 2] No such file or directory: '/tmp/pip-o6Tpui-build/setup.py'
+```
+
+Solution: upgrade your version of `pip`:
+
+```bash
+pip install --upgrade pip
+```
+
+This may require `sudo`, depending on how `pip` is installed.
+
+#### SSLError: SSL_VERIFY_FAILED <a class="md-anchor" id="AUTOGENERATED-sslerror--ssl_verify_failed"></a>
+
+If, during pip install from a URL, you encounter an error like:
+
+```bash
+...
+SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
+```
+
+Solution: Download the wheel manually via curl or wget, and pip install locally.
+
+### On Linux <a class="md-anchor" id="AUTOGENERATED-on-linux"></a>
+
+If you encounter:
+
+```python
+...
+ "__add__", "__radd__",
+             ^
+SyntaxError: invalid syntax
+```
+
+Solution: make sure you are using Python 2.7.
+
+### On MacOSX <a class="md-anchor" id="AUTOGENERATED-on-macosx"></a>
+
+
+If you encounter:
+
+```python
+import six.moves.copyreg as copyreg
+
+ImportError: No module named copyreg
+```
+
+Solution: TensorFlow depends on protobuf, which requires `six-1.10.0`. Apple's
+default python environment has `six-1.4.1` and may be difficult to upgrade.
+There are several ways to fix this:
+
+1. Upgrade the system-wide copy of `six`:
+
+    ```bash
+    sudo easy_install -U six
+    ```
+
+2. Install a separate copy of python via homebrew:
+
+    ```bash
+    brew install python
+    ```
+
+3. Build or use TensorFlow
+   [within `virtualenv`](#virtualenv_install).
+
+
+
+If you encounter:
+
+```
+>>> import tensorflow as tf
+Traceback (most recent call last):
+  File "<stdin>", line 1, in <module>
+  File "/usr/local/lib/python2.7/site-packages/tensorflow/__init__.py", line 4, in <module>
+    from tensorflow.python import *
+  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 13, in <module>
+    from tensorflow.core.framework.graph_pb2 import *
+...
+  File "/usr/local/lib/python2.7/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 22, in <module>
+    serialized_pb=_b('\n,tensorflow/core/framework/tensor_shape.proto\x12\ntensorflow\"d\n\x10TensorShapeProto\x12-\n\x03\x64im\x18\x02 \x03(\x0b\x32 .tensorflow.TensorShapeProto.Dim\x1a!\n\x03\x44im\x12\x0c\n\x04size\x18\x01 \x01(\x03\x12\x0c\n\x04name\x18\x02 \x01(\tb\x06proto3')
+TypeError: __init__() got an unexpected keyword argument 'syntax'
+```
+
+This is due to a conflict between protobuf versions (we require protobuf 3.0.0).
+The best current solution is to make sure older versions of protobuf are not
+installed, such as:
+
+```bash
+brew reinstall --devel protobuf
+```

examples/2 - Basic Classifiers/input_data.py

@@ -0,0 +1,144 @@
+"""Functions for downloading and reading MNIST data."""
+from __future__ import print_function
+import gzip
+import os
+import urllib
+import numpy
+SOURCE_URL = 'http://yann.lecun.com/exdb/mnist/'
+def maybe_download(filename, work_directory):
+  """Download the data from Yann's website, unless it's already here."""
+  if not os.path.exists(work_directory):
+    os.mkdir(work_directory)
+  filepath = os.path.join(work_directory, filename)
+  if not os.path.exists(filepath):
+    filepath, _ = urllib.urlretrieve(SOURCE_URL + filename, filepath)
+    statinfo = os.stat(filepath)
+    print('Succesfully downloaded', filename, statinfo.st_size, 'bytes.')
+  return filepath
+def _read32(bytestream):
+  dt = numpy.dtype(numpy.uint32).newbyteorder('>')
+  return numpy.frombuffer(bytestream.read(4), dtype=dt)
+def extract_images(filename):
+  """Extract the images into a 4D uint8 numpy array [index, y, x, depth]."""
+  print('Extracting', filename)
+  with gzip.open(filename) as bytestream:
+    magic = _read32(bytestream)
+    if magic != 2051:
+      raise ValueError(
+          'Invalid magic number %d in MNIST image file: %s' %
+          (magic, filename))
+    num_images = _read32(bytestream)
+    rows = _read32(bytestream)
+    cols = _read32(bytestream)
+    buf = bytestream.read(rows * cols * num_images)
+    data = numpy.frombuffer(buf, dtype=numpy.uint8)
+    data = data.reshape(num_images, rows, cols, 1)
+    return data
+def dense_to_one_hot(labels_dense, num_classes=10):
+  """Convert class labels from scalars to one-hot vectors."""
+  num_labels = labels_dense.shape[0]
+  index_offset = numpy.arange(num_labels) * num_classes
+  labels_one_hot = numpy.zeros((num_labels, num_classes))
+  labels_one_hot.flat[index_offset + labels_dense.ravel()] = 1
+  return labels_one_hot
+def extract_labels(filename, one_hot=False):
+  """Extract the labels into a 1D uint8 numpy array [index]."""
+  print('Extracting', filename)
+  with gzip.open(filename) as bytestream:
+    magic = _read32(bytestream)
+    if magic != 2049:
+      raise ValueError(
+          'Invalid magic number %d in MNIST label file: %s' %
+          (magic, filename))
+    num_items = _read32(bytestream)
+    buf = bytestream.read(num_items)
+    labels = numpy.frombuffer(buf, dtype=numpy.uint8)
+    if one_hot:
+      return dense_to_one_hot(labels)
+    return labels
+class DataSet(object):
+  def __init__(self, images, labels, fake_data=False):
+    if fake_data:
+      self._num_examples = 10000
+    else:
+      assert images.shape[0] == labels.shape[0], (
+          "images.shape: %s labels.shape: %s" % (images.shape,
+                                                 labels.shape))
+      self._num_examples = images.shape[0]
+      # Convert shape from [num examples, rows, columns, depth]
+      # to [num examples, rows*columns] (assuming depth == 1)
+      assert images.shape[3] == 1
+      images = images.reshape(images.shape[0],
+                              images.shape[1] * images.shape[2])
+      # Convert from [0, 255] -> [0.0, 1.0].
+      images = images.astype(numpy.float32)
+      images = numpy.multiply(images, 1.0 / 255.0)
+    self._images = images
+    self._labels = labels
+    self._epochs_completed = 0
+    self._index_in_epoch = 0
+  @property
+  def images(self):
+    return self._images
+  @property
+  def labels(self):
+    return self._labels
+  @property
+  def num_examples(self):
+    return self._num_examples
+  @property
+  def epochs_completed(self):
+    return self._epochs_completed
+  def next_batch(self, batch_size, fake_data=False):
+    """Return the next `batch_size` examples from this data set."""
+    if fake_data:
+      fake_image = [1.0 for _ in xrange(784)]
+      fake_label = 0
+      return [fake_image for _ in xrange(batch_size)], [
+          fake_label for _ in xrange(batch_size)]
+    start = self._index_in_epoch
+    self._index_in_epoch += batch_size
+    if self._index_in_epoch > self._num_examples:
+      # Finished epoch
+      self._epochs_completed += 1
+      # Shuffle the data
+      perm = numpy.arange(self._num_examples)
+      numpy.random.shuffle(perm)
+      self._images = self._images[perm]
+      self._labels = self._labels[perm]
+      # Start next epoch
+      start = 0
+      self._index_in_epoch = batch_size
+      assert batch_size <= self._num_examples
+    end = self._index_in_epoch
+    return self._images[start:end], self._labels[start:end]
+def read_data_sets(train_dir, fake_data=False, one_hot=False):
+  class DataSets(object):
+    pass
+  data_sets = DataSets()
+  if fake_data:
+    data_sets.train = DataSet([], [], fake_data=True)
+    data_sets.validation = DataSet([], [], fake_data=True)
+    data_sets.test = DataSet([], [], fake_data=True)
+    return data_sets
+  TRAIN_IMAGES = 'train-images-idx3-ubyte.gz'
+  TRAIN_LABELS = 'train-labels-idx1-ubyte.gz'
+  TEST_IMAGES = 't10k-images-idx3-ubyte.gz'
+  TEST_LABELS = 't10k-labels-idx1-ubyte.gz'
+  VALIDATION_SIZE = 5000
+  local_file = maybe_download(TRAIN_IMAGES, train_dir)
+  train_images = extract_images(local_file)
+  local_file = maybe_download(TRAIN_LABELS, train_dir)
+  train_labels = extract_labels(local_file, one_hot=one_hot)
+  local_file = maybe_download(TEST_IMAGES, train_dir)
+  test_images = extract_images(local_file)
+  local_file = maybe_download(TEST_LABELS, train_dir)
+  test_labels = extract_labels(local_file, one_hot=one_hot)
+  validation_images = train_images[:VALIDATION_SIZE]
+  validation_labels = train_labels[:VALIDATION_SIZE]
+  train_images = train_images[VALIDATION_SIZE:]
+  train_labels = train_labels[VALIDATION_SIZE:]
+  data_sets.train = DataSet(train_images, train_labels)
+  data_sets.validation = DataSet(validation_images, validation_labels)
+  data_sets.test = DataSet(test_images, test_labels)
+  return data_sets

alexnet.py

@@ -126,7 +126,7 @@ with tf.Session() as sess:
             acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})
             # Calculate batch loss
             loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})
-            print "Iter " + str(step*batch_size) + ", Loss= " + "{:.6f}".format(loss) + ", Training Accuracy= " + "{:.5f}".format(acc)
+            print "Iter " + str(step*batch_size) + ", Minibatch Loss= " + "{:.6f}".format(loss) + ", Training Accuracy= " + "{:.5f}".format(acc)
         step += 1
     print "Optimization Finished!"
     # Calculate accuracy for 256 mnist test images

convolutional_network.py

@@ -1,5 +1,5 @@
 '''
-A Multilayer Perceptron implementation example using TensorFlow library.
+A Convolutional Network implementation example using TensorFlow library.
 This example is using the MNIST database of handwritten digits (http://yann.lecun.com/exdb/mnist/)
 
 Author: Aymeric Damien
@@ -105,7 +105,7 @@ with tf.Session() as sess:
             acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})
             # Calculate batch loss
             loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})
-            print "Iter " + str(step*batch_size) + ", Loss= " + "{:.6f}".format(loss) + ", Training Accuracy= " + "{:.5f}".format(acc)
+            print "Iter " + str(step*batch_size) + ", Minibatch Loss= " + "{:.6f}".format(loss) + ", Training Accuracy= " + "{:.5f}".format(acc)
         step += 1
     print "Optimization Finished!"
     # Calculate accuracy for 256 mnist test images

examples/3 - Neural Networks/input_data.py

@@ -0,0 +1,144 @@
+"""Functions for downloading and reading MNIST data."""
+from __future__ import print_function
+import gzip
+import os
+import urllib
+import numpy
+SOURCE_URL = 'http://yann.lecun.com/exdb/mnist/'
+def maybe_download(filename, work_directory):
+  """Download the data from Yann's website, unless it's already here."""
+  if not os.path.exists(work_directory):
+    os.mkdir(work_directory)
+  filepath = os.path.join(work_directory, filename)
+  if not os.path.exists(filepath):
+    filepath, _ = urllib.urlretrieve(SOURCE_URL + filename, filepath)
+    statinfo = os.stat(filepath)
+    print('Succesfully downloaded', filename, statinfo.st_size, 'bytes.')
+  return filepath
+def _read32(bytestream):
+  dt = numpy.dtype(numpy.uint32).newbyteorder('>')
+  return numpy.frombuffer(bytestream.read(4), dtype=dt)
+def extract_images(filename):
+  """Extract the images into a 4D uint8 numpy array [index, y, x, depth]."""
+  print('Extracting', filename)
+  with gzip.open(filename) as bytestream:
+    magic = _read32(bytestream)
+    if magic != 2051:
+      raise ValueError(
+          'Invalid magic number %d in MNIST image file: %s' %
+          (magic, filename))
+    num_images = _read32(bytestream)
+    rows = _read32(bytestream)
+    cols = _read32(bytestream)
+    buf = bytestream.read(rows * cols * num_images)
+    data = numpy.frombuffer(buf, dtype=numpy.uint8)
+    data = data.reshape(num_images, rows, cols, 1)
+    return data
+def dense_to_one_hot(labels_dense, num_classes=10):
+  """Convert class labels from scalars to one-hot vectors."""
+  num_labels = labels_dense.shape[0]
+  index_offset = numpy.arange(num_labels) * num_classes
+  labels_one_hot = numpy.zeros((num_labels, num_classes))
+  labels_one_hot.flat[index_offset + labels_dense.ravel()] = 1
+  return labels_one_hot
+def extract_labels(filename, one_hot=False):
+  """Extract the labels into a 1D uint8 numpy array [index]."""
+  print('Extracting', filename)
+  with gzip.open(filename) as bytestream:
+    magic = _read32(bytestream)
+    if magic != 2049:
+      raise ValueError(
+          'Invalid magic number %d in MNIST label file: %s' %
+          (magic, filename))
+    num_items = _read32(bytestream)
+    buf = bytestream.read(num_items)
+    labels = numpy.frombuffer(buf, dtype=numpy.uint8)
+    if one_hot:
+      return dense_to_one_hot(labels)
+    return labels
+class DataSet(object):
+  def __init__(self, images, labels, fake_data=False):
+    if fake_data:
+      self._num_examples = 10000
+    else:
+      assert images.shape[0] == labels.shape[0], (
+          "images.shape: %s labels.shape: %s" % (images.shape,
+                                                 labels.shape))
+      self._num_examples = images.shape[0]
+      # Convert shape from [num examples, rows, columns, depth]
+      # to [num examples, rows*columns] (assuming depth == 1)
+      assert images.shape[3] == 1
+      images = images.reshape(images.shape[0],
+                              images.shape[1] * images.shape[2])
+      # Convert from [0, 255] -> [0.0, 1.0].
+      images = images.astype(numpy.float32)
+      images = numpy.multiply(images, 1.0 / 255.0)
+    self._images = images
+    self._labels = labels
+    self._epochs_completed = 0
+    self._index_in_epoch = 0
+  @property
+  def images(self):
+    return self._images
+  @property
+  def labels(self):
+    return self._labels
+  @property
+  def num_examples(self):
+    return self._num_examples
+  @property
+  def epochs_completed(self):
+    return self._epochs_completed
+  def next_batch(self, batch_size, fake_data=False):
+    """Return the next `batch_size` examples from this data set."""
+    if fake_data:
+      fake_image = [1.0 for _ in xrange(784)]
+      fake_label = 0
+      return [fake_image for _ in xrange(batch_size)], [
+          fake_label for _ in xrange(batch_size)]
+    start = self._index_in_epoch
+    self._index_in_epoch += batch_size
+    if self._index_in_epoch > self._num_examples:
+      # Finished epoch
+      self._epochs_completed += 1
+      # Shuffle the data
+      perm = numpy.arange(self._num_examples)
+      numpy.random.shuffle(perm)
+      self._images = self._images[perm]
+      self._labels = self._labels[perm]
+      # Start next epoch
+      start = 0
+      self._index_in_epoch = batch_size
+      assert batch_size <= self._num_examples
+    end = self._index_in_epoch
+    return self._images[start:end], self._labels[start:end]
+def read_data_sets(train_dir, fake_data=False, one_hot=False):
+  class DataSets(object):
+    pass
+  data_sets = DataSets()
+  if fake_data:
+    data_sets.train = DataSet([], [], fake_data=True)
+    data_sets.validation = DataSet([], [], fake_data=True)
+    data_sets.test = DataSet([], [], fake_data=True)
+    return data_sets
+  TRAIN_IMAGES = 'train-images-idx3-ubyte.gz'
+  TRAIN_LABELS = 'train-labels-idx1-ubyte.gz'
+  TEST_IMAGES = 't10k-images-idx3-ubyte.gz'
+  TEST_LABELS = 't10k-labels-idx1-ubyte.gz'
+  VALIDATION_SIZE = 5000
+  local_file = maybe_download(TRAIN_IMAGES, train_dir)
+  train_images = extract_images(local_file)
+  local_file = maybe_download(TRAIN_LABELS, train_dir)
+  train_labels = extract_labels(local_file, one_hot=one_hot)
+  local_file = maybe_download(TEST_IMAGES, train_dir)
+  test_images = extract_images(local_file)
+  local_file = maybe_download(TEST_LABELS, train_dir)
+  test_labels = extract_labels(local_file, one_hot=one_hot)
+  validation_images = train_images[:VALIDATION_SIZE]
+  validation_labels = train_labels[:VALIDATION_SIZE]
+  train_images = train_images[VALIDATION_SIZE:]
+  train_labels = train_labels[VALIDATION_SIZE:]
+  data_sets.train = DataSet(train_images, train_labels)
+  data_sets.validation = DataSet(validation_images, validation_labels)
+  data_sets.test = DataSet(test_images, test_labels)
+  return data_sets

multilayer_perceptron.py

@@ -18,7 +18,7 @@ training_epochs = 15
 batch_size = 100
 display_step = 1
 
-#Network Parameters
+# Network Parameters
 n_hidden_1 = 256 # 1st layer num features
 n_hidden_2 = 256 # 2nd layer num features
 n_input = 784 # MNIST data input (img shape: 28*28)

examples/4 - Multi GPU/multigpu_basics.py

@@ -0,0 +1,91 @@
+'''
+Basic Multi GPU computation example using TensorFlow library.
+
+Author: Aymeric Damien
+Project: https://github.com/aymericdamien/TensorFlow-Examples/
+'''
+
+'''
+This tutorial requires your machine to have 2 GPUs
+"/cpu:0": The CPU of your machine.
+"/gpu:0": The first GPU of your machine
+"/gpu:1": The second GPU of your machine
+'''
+
+import numpy as np
+import tensorflow as tf
+import datetime
+
+#Processing Units logs
+log_device_placement = True
+
+#num of multiplications to perform
+n = 10
+
+'''
+Example: compute A^n + B^n on 2 GPUs
+Results on 8 cores with 2 GTX-980:
+ * Single GPU computation time: 0:00:11.277449
+ * Multi GPU computation time: 0:00:07.131701
+'''
+#Create random large matrix
+A = np.random.rand(1e4, 1e4).astype('float32')
+B = np.random.rand(1e4, 1e4).astype('float32')
+
+# Creates a graph to store results
+c1 = []
+c2 = []
+
+def matpow(M, n):
+    if n < 1: #Abstract cases where n < 1
+        return M
+    else:
+        return tf.matmul(M, matpow(M, n-1))
+
+'''
+Single GPU computing
+'''
+with tf.device('/gpu:0'):
+    a = tf.constant(A)
+    b = tf.constant(B)
+    #compute A^n and B^n and store results in c1
+    c1.append(matpow(a, n))
+    c1.append(matpow(b, n))
+
+with tf.device('/cpu:0'):
+  sum = tf.add_n(c1) #Addition of all elements in c1, i.e. A^n + B^n
+
+t1_1 = datetime.datetime.now()
+with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:
+    # Runs the op.
+    sess.run(sum)
+t2_1 = datetime.datetime.now()
+
+
+'''
+Multi GPU computing
+'''
+#GPU:0 computes A^n
+with tf.device('/gpu:0'):
+    #compute A^n and store result in c2
+    a = tf.constant(A)
+    c2.append(matpow(a, n))
+
+#GPU:1 computes B^n
+with tf.device('/gpu:1'):
+    #compute B^n and store result in c2
+    b = tf.constant(B)
+    c2.append(matpow(b, n))
+
+with tf.device('/cpu:0'):
+  sum = tf.add_n(c2) #Addition of all elements in c2, i.e. A^n + B^n
+
+t1_2 = datetime.datetime.now()
+with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:
+    # Runs the op.
+    sess.run(sum)
+t2_2 = datetime.datetime.now()
+
+
+print "Single GPU computation time: " + str(t2_1-t1_1)
+print "Multi GPU computation time: " + str(t2_2-t1_2)

notebooks/1 - Introduction/basic_operations.ipynb

@@ -0,0 +1,220 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Basic Operations example using TensorFlow library.\n",
+    "# Author: Aymeric Damien\n",
+    "# Project: https://github.com/aymericdamien/TensorFlow-Examples/"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import tensorflow as tf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Basic constant operations\n",
+    "# The value returned by the constructor represents the output\n",
+    "# of the Constant op.\n",
+    "a = tf.constant(2)\n",
+    "b = tf.constant(3)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "a=2, b=3\n",
+      "Addition with constants: 5\n",
+      "Multiplication with constants: 6\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Launch the default graph.\n",
+    "with tf.Session() as sess:\n",
+    "    print \"a=2, b=3\"\n",
+    "    print \"Addition with constants: %i\" % sess.run(a+b)\n",
+    "    print \"Multiplication with constants: %i\" % sess.run(a*b)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Basic Operations with variable as graph input\n",
+    "# The value returned by the constructor represents the output\n",
+    "# of the Variable op. (define as input when running session)\n",
+    "# tf Graph input\n",
+    "a = tf.placeholder(tf.types.int16)\n",
+    "b = tf.placeholder(tf.types.int16)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Define some operations\n",
+    "add = tf.add(a, b)\n",
+    "mul = tf.mul(a, b)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Addition with variables: 5\n",
+      "Multiplication with variables: 6\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Launch the default graph.\n",
+    "with tf.Session() as sess:\n",
+    "    # Run every operation with variable input\n",
+    "    print \"Addition with variables: %i\" % sess.run(add, feed_dict={a: 2, b: 3})\n",
+    "    print \"Multiplication with variables: %i\" % sess.run(mul, feed_dict={a: 2, b: 3})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# ----------------\n",
+    "# More in details:\n",
+    "# Matrix Multiplication from TensorFlow official tutorial\n",
+    "\n",
+    "# Create a Constant op that produces a 1x2 matrix.  The op is\n",
+    "# added as a node to the default graph.\n",
+    "#\n",
+    "# The value returned by the constructor represents the output\n",
+    "# of the Constant op.\n",
+    "matrix1 = tf.constant([[3., 3.]])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Create another Constant that produces a 2x1 matrix.\n",
+    "matrix2 = tf.constant([[2.],[2.]])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Create a Matmul op that takes 'matrix1' and 'matrix2' as inputs.\n",
+    "# The returned value, 'product', represents the result of the matrix\n",
+    "# multiplication.\n",
+    "product = tf.matmul(matrix1, matrix2)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[[ 12.]]\n"
+     ]
+    }
+   ],
+   "source": [
+    "# To run the matmul op we call the session 'run()' method, passing 'product'\n",
+    "# which represents the output of the matmul op.  This indicates to the call\n",
+    "# that we want to get the output of the matmul op back.\n",
+    "#\n",
+    "# All inputs needed by the op are run automatically by the session.  They\n",
+    "# typically are run in parallel.\n",
+    "#\n",
+    "# The call 'run(product)' thus causes the execution of threes ops in the\n",
+    "# graph: the two constants and matmul.\n",
+    "#\n",
+    "# The output of the op is returned in 'result' as a numpy `ndarray` object.\n",
+    "with tf.Session() as sess:\n",
+    "    result = sess.run(product)\n",
+    "    print result"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "IPython (Python 2.7)",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

notebooks/1 - Introduction/helloworld.ipynb

@@ -0,0 +1,87 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import tensorflow as tf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Simple hello world using TensorFlow\n",
+    "\n",
+    "# Create a Constant op\n",
+    "# The op is added as a node to the default graph.\n",
+    "#\n",
+    "# The value returned by the constructor represents the output\n",
+    "# of the Constant op.\n",
+    "\n",
+    "hello = tf.constant('Hello, TensorFlow!')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Start tf session\n",
+    "sess = tf.Session()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hello, TensorFlow!\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Run graph\n",
+    "print sess.run(hello)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "IPython (Python 2.7)",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

notebooks/2 - Basic Classifiers/logistic_regression.ipynb

@@ -0,0 +1,228 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# A logistic regression learning algorithm example using TensorFlow library.\n",
+    "# This example is using the MNIST database of handwritten digits \n",
+    "# (http://yann.lecun.com/exdb/mnist/)\n",
+    "\n",
+    "# Author: Aymeric Damien\n",
+    "# Project: https://github.com/aymericdamien/TensorFlow-Examples/"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Import MINST data\n",
+    "import input_data\n",
+    "mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import tensorflow as tf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Parameters\n",
+    "learning_rate = 0.01\n",
+    "training_epochs = 25\n",
+    "batch_size = 100\n",
+    "display_step = 1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# tf Graph Input\n",
+    "x = tf.placeholder(\"float\", [None, 784]) # mnist data image of shape 28*28=784\n",
+    "y = tf.placeholder(\"float\", [None, 10]) # 0-9 digits recognition => 10 classes"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Create model\n",
+    "\n",
+    "# Set model weights\n",
+    "W = tf.Variable(tf.zeros([784, 10]))\n",
+    "b = tf.Variable(tf.zeros([10]))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Construct model\n",
+    "activation = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Minimize error using cross entropy\n",
+    "# Cross entropy\n",
+    "cost = -tf.reduce_sum(y*tf.log(activation)) \n",
+    "# Gradient Descent\n",
+    "optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Initializing the variables\n",
+    "init = tf.initialize_all_variables()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch: 0001 cost= 29.860479714\n",
+      "Epoch: 0002 cost= 22.080549484\n",
+      "Epoch: 0003 cost= 21.237104595\n",
+      "Epoch: 0004 cost= 20.460196280\n",
+      "Epoch: 0005 cost= 20.185128237\n",
+      "Epoch: 0006 cost= 19.940297202\n",
+      "Epoch: 0007 cost= 19.645111119\n",
+      "Epoch: 0008 cost= 19.507218031\n",
+      "Epoch: 0009 cost= 19.389794492\n",
+      "Epoch: 0010 cost= 19.177005816\n",
+      "Epoch: 0011 cost= 19.082493615\n",
+      "Epoch: 0012 cost= 19.072873598\n",
+      "Epoch: 0013 cost= 18.938005402\n",
+      "Epoch: 0014 cost= 18.891806430\n",
+      "Epoch: 0015 cost= 18.839480221\n",
+      "Epoch: 0016 cost= 18.769349510\n",
+      "Epoch: 0017 cost= 18.590865587\n",
+      "Epoch: 0018 cost= 18.623413677\n",
+      "Epoch: 0019 cost= 18.546149085\n",
+      "Epoch: 0020 cost= 18.432274895\n",
+      "Epoch: 0021 cost= 18.358189004\n",
+      "Epoch: 0022 cost= 18.380014628\n",
+      "Epoch: 0023 cost= 18.499993471\n",
+      "Epoch: 0024 cost= 18.386477311\n",
+      "Epoch: 0025 cost= 18.258080609\n",
+      "Optimization Finished!\n",
+      "Accuracy: 0.9048\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Launch the graph\n",
+    "with tf.Session() as sess:\n",
+    "    sess.run(init)\n",
+    "\n",
+    "    # Training cycle\n",
+    "    for epoch in range(training_epochs):\n",
+    "        avg_cost = 0.\n",
+    "        total_batch = int(mnist.train.num_examples/batch_size)\n",
+    "        # Loop over all batches\n",
+    "        for i in range(total_batch):\n",
+    "            batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
+    "            # Fit training using batch data\n",
+    "            sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys})\n",
+    "            # Compute average loss\n",
+    "            avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys})/total_batch\n",
+    "        # Display logs per epoch step\n",
+    "        if epoch % display_step == 0:\n",
+    "            print \"Epoch:\", '%04d' % (epoch+1), \"cost=\", \"{:.9f}\".format(avg_cost)\n",
+    "\n",
+    "    print \"Optimization Finished!\"\n",
+    "\n",
+    "    # Test model\n",
+    "    correct_prediction = tf.equal(tf.argmax(activation, 1), tf.argmax(y, 1))\n",
+    "    # Calculate accuracy\n",
+    "    accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
+    "    print \"Accuracy:\", accuracy.eval({x: mnist.test.images, y: mnist.test.labels})"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "IPython (Python 2.7)",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

notebooks/2 - Basic Classifiers/nearest_neighbor.ipynb

@@ -0,0 +1,389 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# A nearest neighbor learning algorithm example using TensorFlow library.\n",
+    "# This example is using the MNIST database of handwritten digits \n",
+    "# (http://yann.lecun.com/exdb/mnist/)\n",
+    "\n",
+    "# Author: Aymeric Damien\n",
+    "# Project: https://github.com/aymericdamien/TensorFlow-Examples/"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import tensorflow as tf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Import MINST data\n",
+    "import input_data\n",
+    "mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# In this example, we limit mnist data\n",
+    "Xtr, Ytr = mnist.train.next_batch(5000) #5000 for training (nn candidates)\n",
+    "Xte, Yte = mnist.test.next_batch(200) #200 for testing"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Reshape images to 1D\n",
+    "Xtr = np.reshape(Xtr, newshape=(-1, 28*28))\n",
+    "Xte = np.reshape(Xte, newshape=(-1, 28*28))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# tf Graph Input\n",
+    "xtr = tf.placeholder(\"float\", [None, 784])\n",
+    "xte = tf.placeholder(\"float\", [784])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Nearest Neighbor calculation using L1 Distance\n",
+    "# Calculate L1 Distance\n",
+    "distance = tf.reduce_sum(tf.abs(tf.add(xtr, tf.neg(xte))), reduction_indices=1)\n",
+    "# Predict: Get min distance index (Nearest neighbor)\n",
+    "pred = tf.arg_min(distance, 0)\n",
+    "\n",
+    "accuracy = 0."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Initializing the variables\n",
+    "init = tf.initialize_all_variables()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Test 0 Prediction: 7 True Class: 7\n",
+      "Test 1 Prediction: 2 True Class: 2\n",
+      "Test 2 Prediction: 1 True Class: 1\n",
+      "Test 3 Prediction: 0 True Class: 0\n",
+      "Test 4 Prediction: 4 True Class: 4\n",
+      "Test 5 Prediction: 1 True Class: 1\n",
+      "Test 6 Prediction: 4 True Class: 4\n",
+      "Test 7 Prediction: 9 True Class: 9\n",
+      "Test 8 Prediction: 8 True Class: 5\n",
+      "Test 9 Prediction: 9 True Class: 9\n",
+      "Test 10 Prediction: 0 True Class: 0\n",
+      "Test 11 Prediction: 0 True Class: 6\n",
+      "Test 12 Prediction: 9 True Class: 9\n",
+      "Test 13 Prediction: 0 True Class: 0\n",
+      "Test 14 Prediction: 1 True Class: 1\n",
+      "Test 15 Prediction: 5 True Class: 5\n",
+      "Test 16 Prediction: 4 True Class: 9\n",
+      "Test 17 Prediction: 7 True Class: 7\n",
+      "Test 18 Prediction: 3 True Class: 3\n",
+      "Test 19 Prediction: 4 True Class: 4\n",
+      "Test 20 Prediction: 9 True Class: 9\n",
+      "Test 21 Prediction: 6 True Class: 6\n",
+      "Test 22 Prediction: 6 True Class: 6\n",
+      "Test 23 Prediction: 5 True Class: 5\n",
+      "Test 24 Prediction: 4 True Class: 4\n",
+      "Test 25 Prediction: 0 True Class: 0\n",
+      "Test 26 Prediction: 7 True Class: 7\n",
+      "Test 27 Prediction: 4 True Class: 4\n",
+      "Test 28 Prediction: 0 True Class: 0\n",
+      "Test 29 Prediction: 1 True Class: 1\n",
+      "Test 30 Prediction: 3 True Class: 3\n",
+      "Test 31 Prediction: 1 True Class: 1\n",
+      "Test 32 Prediction: 3 True Class: 3\n",
+      "Test 33 Prediction: 4 True Class: 4\n",
+      "Test 34 Prediction: 7 True Class: 7\n",
+      "Test 35 Prediction: 2 True Class: 2\n",
+      "Test 36 Prediction: 7 True Class: 7\n",
+      "Test 37 Prediction: 1 True Class: 1\n",
+      "Test 38 Prediction: 2 True Class: 2\n",
+      "Test 39 Prediction: 1 True Class: 1\n",
+      "Test 40 Prediction: 1 True Class: 1\n",
+      "Test 41 Prediction: 7 True Class: 7\n",
+      "Test 42 Prediction: 4 True Class: 4\n",
+      "Test 43 Prediction: 1 True Class: 2\n",
+      "Test 44 Prediction: 3 True Class: 3\n",
+      "Test 45 Prediction: 5 True Class: 5\n",
+      "Test 46 Prediction: 1 True Class: 1\n",
+      "Test 47 Prediction: 2 True Class: 2\n",
+      "Test 48 Prediction: 4 True Class: 4\n",
+      "Test 49 Prediction: 4 True Class: 4\n",
+      "Test 50 Prediction: 6 True Class: 6\n",
+      "Test 51 Prediction: 3 True Class: 3\n",
+      "Test 52 Prediction: 5 True Class: 5\n",
+      "Test 53 Prediction: 5 True Class: 5\n",
+      "Test 54 Prediction: 6 True Class: 6\n",
+      "Test 55 Prediction: 0 True Class: 0\n",
+      "Test 56 Prediction: 4 True Class: 4\n",
+      "Test 57 Prediction: 1 True Class: 1\n",
+      "Test 58 Prediction: 9 True Class: 9\n",
+      "Test 59 Prediction: 5 True Class: 5\n",
+      "Test 60 Prediction: 7 True Class: 7\n",
+      "Test 61 Prediction: 8 True Class: 8\n",
+      "Test 62 Prediction: 9 True Class: 9\n",
+      "Test 63 Prediction: 3 True Class: 3\n",
+      "Test 64 Prediction: 7 True Class: 7\n",
+      "Test 65 Prediction: 4 True Class: 4\n",
+      "Test 66 Prediction: 6 True Class: 6\n",
+      "Test 67 Prediction: 4 True Class: 4\n",
+      "Test 68 Prediction: 3 True Class: 3\n",
+      "Test 69 Prediction: 0 True Class: 0\n",
+      "Test 70 Prediction: 7 True Class: 7\n",
+      "Test 71 Prediction: 0 True Class: 0\n",
+      "Test 72 Prediction: 2 True Class: 2\n",
+      "Test 73 Prediction: 7 True Class: 9\n",
+      "Test 74 Prediction: 1 True Class: 1\n",
+      "Test 75 Prediction: 7 True Class: 7\n",
+      "Test 76 Prediction: 3 True Class: 3\n",
+      "Test 77 Prediction: 7 True Class: 2\n",
+      "Test 78 Prediction: 9 True Class: 9\n",
+      "Test 79 Prediction: 7 True Class: 7\n",
+      "Test 80 Prediction: 7 True Class: 7\n",
+      "Test 81 Prediction: 6 True Class: 6\n",
+      "Test 82 Prediction: 2 True Class: 2\n",
+      "Test 83 Prediction: 7 True Class: 7\n",
+      "Test 84 Prediction: 8 True Class: 8\n",
+      "Test 85 Prediction: 4 True Class: 4\n",
+      "Test 86 Prediction: 7 True Class: 7\n",
+      "Test 87 Prediction: 3 True Class: 3\n",
+      "Test 88 Prediction: 6 True Class: 6\n",
+      "Test 89 Prediction: 1 True Class: 1\n",
+      "Test 90 Prediction: 3 True Class: 3\n",
+      "Test 91 Prediction: 6 True Class: 6\n",
+      "Test 92 Prediction: 9 True Class: 9\n",
+      "Test 93 Prediction: 3 True Class: 3\n",
+      "Test 94 Prediction: 1 True Class: 1\n",
+      "Test 95 Prediction: 4 True Class: 4\n",
+      "Test 96 Prediction: 1 True Class: 1\n",
+      "Test 97 Prediction: 7 True Class: 7\n",
+      "Test 98 Prediction: 6 True Class: 6\n",
+      "Test 99 Prediction: 9 True Class: 9\n",
+      "Test 100 Prediction: 6 True Class: 6\n",
+      "Test 101 Prediction: 0 True Class: 0\n",
+      "Test 102 Prediction: 5 True Class: 5\n",
+      "Test 103 Prediction: 4 True Class: 4\n",
+      "Test 104 Prediction: 9 True Class: 9\n",
+      "Test 105 Prediction: 9 True Class: 9\n",
+      "Test 106 Prediction: 2 True Class: 2\n",
+      "Test 107 Prediction: 1 True Class: 1\n",
+      "Test 108 Prediction: 9 True Class: 9\n",
+      "Test 109 Prediction: 4 True Class: 4\n",
+      "Test 110 Prediction: 8 True Class: 8\n",
+      "Test 111 Prediction: 7 True Class: 7\n",
+      "Test 112 Prediction: 3 True Class: 3\n",
+      "Test 113 Prediction: 9 True Class: 9\n",
+      "Test 114 Prediction: 7 True Class: 7\n",
+      "Test 115 Prediction: 9 True Class: 4\n",
+      "Test 116 Prediction: 9 True Class: 4\n",
+      "Test 117 Prediction: 4 True Class: 4\n",
+      "Test 118 Prediction: 9 True Class: 9\n",
+      "Test 119 Prediction: 7 True Class: 2\n",
+      "Test 120 Prediction: 5 True Class: 5\n",
+      "Test 121 Prediction: 4 True Class: 4\n",
+      "Test 122 Prediction: 7 True Class: 7\n",
+      "Test 123 Prediction: 6 True Class: 6\n",
+      "Test 124 Prediction: 7 True Class: 7\n",
+      "Test 125 Prediction: 9 True Class: 9\n",
+      "Test 126 Prediction: 0 True Class: 0\n",
+      "Test 127 Prediction: 5 True Class: 5\n",
+      "Test 128 Prediction: 8 True Class: 8\n",
+      "Test 129 Prediction: 5 True Class: 5\n",
+      "Test 130 Prediction: 6 True Class: 6\n",
+      "Test 131 Prediction: 6 True Class: 6\n",
+      "Test 132 Prediction: 5 True Class: 5\n",
+      "Test 133 Prediction: 7 True Class: 7\n",
+      "Test 134 Prediction: 8 True Class: 8\n",
+      "Test 135 Prediction: 1 True Class: 1\n",
+      "Test 136 Prediction: 0 True Class: 0\n",
+      "Test 137 Prediction: 1 True Class: 1\n",
+      "Test 138 Prediction: 6 True Class: 6\n",
+      "Test 139 Prediction: 4 True Class: 4\n",
+      "Test 140 Prediction: 6 True Class: 6\n",
+      "Test 141 Prediction: 7 True Class: 7\n",
+      "Test 142 Prediction: 2 True Class: 3\n",
+      "Test 143 Prediction: 1 True Class: 1\n",
+      "Test 144 Prediction: 7 True Class: 7\n",
+      "Test 145 Prediction: 1 True Class: 1\n",
+      "Test 146 Prediction: 8 True Class: 8\n",
+      "Test 147 Prediction: 2 True Class: 2\n",
+      "Test 148 Prediction: 0 True Class: 0\n",
+      "Test 149 Prediction: 1 True Class: 2\n",
+      "Test 150 Prediction: 9 True Class: 9\n",
+      "Test 151 Prediction: 9 True Class: 9\n",
+      "Test 152 Prediction: 5 True Class: 5\n",
+      "Test 153 Prediction: 5 True Class: 5\n",
+      "Test 154 Prediction: 1 True Class: 1\n",
+      "Test 155 Prediction: 5 True Class: 5\n",
+      "Test 156 Prediction: 6 True Class: 6\n",
+      "Test 157 Prediction: 0 True Class: 0\n",
+      "Test 158 Prediction: 3 True Class: 3\n",
+      "Test 159 Prediction: 4 True Class: 4\n",
+      "Test 160 Prediction: 4 True Class: 4\n",
+      "Test 161 Prediction: 6 True Class: 6\n",
+      "Test 162 Prediction: 5 True Class: 5\n",
+      "Test 163 Prediction: 4 True Class: 4\n",
+      "Test 164 Prediction: 6 True Class: 6\n",
+      "Test 165 Prediction: 5 True Class: 5\n",
+      "Test 166 Prediction: 4 True Class: 4\n",
+      "Test 167 Prediction: 5 True Class: 5\n",
+      "Test 168 Prediction: 1 True Class: 1\n",
+      "Test 169 Prediction: 4 True Class: 4\n",
+      "Test 170 Prediction: 9 True Class: 4\n",
+      "Test 171 Prediction: 7 True Class: 7\n",
+      "Test 172 Prediction: 2 True Class: 2\n",
+      "Test 173 Prediction: 3 True Class: 3\n",
+      "Test 174 Prediction: 2 True Class: 2\n",
+      "Test 175 Prediction: 1 True Class: 7\n",
+      "Test 176 Prediction: 1 True Class: 1\n",
+      "Test 177 Prediction: 8 True Class: 8\n",
+      "Test 178 Prediction: 1 True Class: 1\n",
+      "Test 179 Prediction: 8 True Class: 8\n",
+      "Test 180 Prediction: 1 True Class: 1\n",
+      "Test 181 Prediction: 8 True Class: 8\n",
+      "Test 182 Prediction: 5 True Class: 5\n",
+      "Test 183 Prediction: 0 True Class: 0\n",
+      "Test 184 Prediction: 2 True Class: 8\n",
+      "Test 185 Prediction: 9 True Class: 9\n",
+      "Test 186 Prediction: 2 True Class: 2\n",
+      "Test 187 Prediction: 5 True Class: 5\n",
+      "Test 188 Prediction: 0 True Class: 0\n",
+      "Test 189 Prediction: 1 True Class: 1\n",
+      "Test 190 Prediction: 1 True Class: 1\n",
+      "Test 191 Prediction: 1 True Class: 1\n",
+      "Test 192 Prediction: 0 True Class: 0\n",
+      "Test 193 Prediction: 4 True Class: 9\n",
+      "Test 194 Prediction: 0 True Class: 0\n",
+      "Test 195 Prediction: 1 True Class: 3\n",
+      "Test 196 Prediction: 1 True Class: 1\n",
+      "Test 197 Prediction: 6 True Class: 6\n",
+      "Test 198 Prediction: 4 True Class: 4\n",
+      "Test 199 Prediction: 2 True Class: 2\n",
+      "Done!\n",
+      "Accuracy: 0.92\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Launch the graph\n",
+    "with tf.Session() as sess:\n",
+    "    sess.run(init)\n",
+    "\n",
+    "    # loop over test data\n",
+    "    for i in range(len(Xte)):\n",
+    "        # Get nearest neighbor\n",
+    "        nn_index = sess.run(pred, feed_dict={xtr: Xtr, xte: Xte[i,:]})\n",
+    "        # Get nearest neighbor class label and compare it to its true label\n",
+    "        print \"Test\", i, \"Prediction:\", np.argmax(Ytr[nn_index]), \\\n",
+    "              \"True Class:\", np.argmax(Yte[i])\n",
+    "        # Calculate accuracy\n",
+    "        if np.argmax(Ytr[nn_index]) == np.argmax(Yte[i]):\n",
+    "            accuracy += 1./len(Xte)\n",
+    "    print \"Done!\"\n",
+    "    print \"Accuracy:\", accuracy"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "IPython (Python 2.7)",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

notebooks/3 - Neural Networks/alexnet.ipynb

@@ -0,0 +1,348 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# AlexNet implementation example using TensorFlow library.\n",
+    "# This example is using the MNIST database of handwritten digits (http://yann.lecun.com/exdb/mnist/)\n",
+    "# AlexNet Paper (http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf)\n",
+    "\n",
+    "# Author: Aymeric Damien\n",
+    "# Project: https://github.com/aymericdamien/TensorFlow-Examples/"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Import MINST data\n",
+    "import input_data\n",
+    "mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import tensorflow as tf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Parameters\n",
+    "learning_rate = 0.001\n",
+    "training_iters = 300000\n",
+    "batch_size = 64\n",
+    "display_step = 100"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Network Parameters\n",
+    "n_input = 784 # MNIST data input (img shape: 28*28)\n",
+    "n_classes = 10 # MNIST total classes (0-9 digits)\n",
+    "dropout = 0.8 # Dropout, probability to keep units"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# tf Graph input\n",
+    "x = tf.placeholder(tf.types.float32, [None, n_input])\n",
+    "y = tf.placeholder(tf.types.float32, [None, n_classes])\n",
+    "keep_prob = tf.placeholder(tf.types.float32) # dropout (keep probability)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Create AlexNet model\n",
+    "def conv2d(name, l_input, w, b):\n",
+    "    return tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(l_input, w, strides=[1, 1, 1, 1], \n",
+    "                                                  padding='SAME'),b), name=name)\n",
+    "\n",
+    "def max_pool(name, l_input, k):\n",
+    "    return tf.nn.max_pool(l_input, ksize=[1, k, k, 1], strides=[1, k, k, 1], \n",
+    "                          padding='SAME', name=name)\n",
+    "\n",
+    "def norm(name, l_input, lsize=4):\n",
+    "    return tf.nn.lrn(l_input, lsize, bias=1.0, alpha=0.001 / 9.0, beta=0.75, name=name)\n",
+    "\n",
+    "def alex_net(_X, _weights, _biases, _dropout):\n",
+    "    # Reshape input picture\n",
+    "    _X = tf.reshape(_X, shape=[-1, 28, 28, 1])\n",
+    "\n",
+    "    # Convolution Layer\n",
+    "    conv1 = conv2d('conv1', _X, _weights['wc1'], _biases['bc1'])\n",
+    "    # Max Pooling (down-sampling)\n",
+    "    pool1 = max_pool('pool1', conv1, k=2)\n",
+    "    # Apply Normalization\n",
+    "    norm1 = norm('norm1', pool1, lsize=4)\n",
+    "    # Apply Dropout\n",
+    "    norm1 = tf.nn.dropout(norm1, _dropout)\n",
+    "\n",
+    "    # Convolution Layer\n",
+    "    conv2 = conv2d('conv2', norm1, _weights['wc2'], _biases['bc2'])\n",
+    "    # Max Pooling (down-sampling)\n",
+    "    pool2 = max_pool('pool2', conv2, k=2)\n",
+    "    # Apply Normalization\n",
+    "    norm2 = norm('norm2', pool2, lsize=4)\n",
+    "    # Apply Dropout\n",
+    "    norm2 = tf.nn.dropout(norm2, _dropout)\n",
+    "\n",
+    "    # Convolution Layer\n",
+    "    conv3 = conv2d('conv3', norm2, _weights['wc3'], _biases['bc3'])\n",
+    "    # Max Pooling (down-sampling)\n",
+    "    pool3 = max_pool('pool3', conv3, k=2)\n",
+    "    # Apply Normalization\n",
+    "    norm3 = norm('norm3', pool3, lsize=4)\n",
+    "    # Apply Dropout\n",
+    "    norm3 = tf.nn.dropout(norm3, _dropout)\n",
+    "\n",
+    "    # Fully connected layer\n",
+    "    # Reshape conv3 output to fit dense layer input\n",
+    "    dense1 = tf.reshape(norm3, [-1, _weights['wd1'].get_shape().as_list()[0]]) \n",
+    "    # Relu activation\n",
+    "    dense1 = tf.nn.relu(tf.matmul(dense1, _weights['wd1']) + _biases['bd1'], name='fc1')\n",
+    "    \n",
+    "    # Relu activation\n",
+    "    dense2 = tf.nn.relu(tf.matmul(dense1, _weights['wd2']) + _biases['bd2'], name='fc2') \n",
+    "\n",
+    "    # Output, class prediction\n",
+    "    out = tf.matmul(dense2, _weights['out']) + _biases['out']\n",
+    "    return out"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Store layers weight & bias\n",
+    "weights = {\n",
+    "    'wc1': tf.Variable(tf.random_normal([3, 3, 1, 64])),\n",
+    "    'wc2': tf.Variable(tf.random_normal([3, 3, 64, 128])),\n",
+    "    'wc3': tf.Variable(tf.random_normal([3, 3, 128, 256])),\n",
+    "    'wd1': tf.Variable(tf.random_normal([4*4*256, 1024])),\n",
+    "    'wd2': tf.Variable(tf.random_normal([1024, 1024])),\n",
+    "    'out': tf.Variable(tf.random_normal([1024, 10]))\n",
+    "}\n",
+    "biases = {\n",
+    "    'bc1': tf.Variable(tf.random_normal([64])),\n",
+    "    'bc2': tf.Variable(tf.random_normal([128])),\n",
+    "    'bc3': tf.Variable(tf.random_normal([256])),\n",
+    "    'bd1': tf.Variable(tf.random_normal([1024])),\n",
+    "    'bd2': tf.Variable(tf.random_normal([1024])),\n",
+    "    'out': tf.Variable(tf.random_normal([n_classes]))\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Construct model\n",
+    "pred = alex_net(x, weights, biases, keep_prob)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Define loss and optimizer\n",
+    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))\n",
+    "optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Evaluate model\n",
+    "correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))\n",
+    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.types.float32))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Initializing the variables\n",
+    "init = tf.initialize_all_variables()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Iter 6400, Minibatch Loss= 29666.185547, Training Accuracy= 0.59375\n",
+      "Iter 12800, Minibatch Loss= 22125.562500, Training Accuracy= 0.60938\n",
+      "Iter 19200, Minibatch Loss= 22631.134766, Training Accuracy= 0.59375\n",
+      "Iter 25600, Minibatch Loss= 18498.414062, Training Accuracy= 0.62500\n",
+      "Iter 32000, Minibatch Loss= 11318.283203, Training Accuracy= 0.70312\n",
+      "Iter 38400, Minibatch Loss= 12076.280273, Training Accuracy= 0.70312\n",
+      "Iter 44800, Minibatch Loss= 8195.520508, Training Accuracy= 0.82812\n",
+      "Iter 51200, Minibatch Loss= 5176.181641, Training Accuracy= 0.84375\n",
+      "Iter 57600, Minibatch Loss= 8951.896484, Training Accuracy= 0.81250\n",
+      "Iter 64000, Minibatch Loss= 10096.946289, Training Accuracy= 0.78125\n",
+      "Iter 70400, Minibatch Loss= 11466.641602, Training Accuracy= 0.68750\n",
+      "Iter 76800, Minibatch Loss= 7469.824219, Training Accuracy= 0.78125\n",
+      "Iter 83200, Minibatch Loss= 4147.449219, Training Accuracy= 0.89062\n",
+      "Iter 89600, Minibatch Loss= 5904.782227, Training Accuracy= 0.82812\n",
+      "Iter 96000, Minibatch Loss= 718.493713, Training Accuracy= 0.93750\n",
+      "Iter 102400, Minibatch Loss= 2184.151367, Training Accuracy= 0.93750\n",
+      "Iter 108800, Minibatch Loss= 2354.463135, Training Accuracy= 0.89062\n",
+      "Iter 115200, Minibatch Loss= 8612.959961, Training Accuracy= 0.81250\n",
+      "Iter 121600, Minibatch Loss= 2225.773926, Training Accuracy= 0.84375\n",
+      "Iter 128000, Minibatch Loss= 160.583618, Training Accuracy= 0.96875\n",
+      "Iter 134400, Minibatch Loss= 1524.846069, Training Accuracy= 0.93750\n",
+      "Iter 140800, Minibatch Loss= 3501.871094, Training Accuracy= 0.89062\n",
+      "Iter 147200, Minibatch Loss= 661.977051, Training Accuracy= 0.96875\n",
+      "Iter 153600, Minibatch Loss= 367.857788, Training Accuracy= 0.98438\n",
+      "Iter 160000, Minibatch Loss= 1735.458740, Training Accuracy= 0.90625\n",
+      "Iter 166400, Minibatch Loss= 209.320374, Training Accuracy= 0.95312\n",
+      "Iter 172800, Minibatch Loss= 1788.553955, Training Accuracy= 0.90625\n",
+      "Iter 179200, Minibatch Loss= 912.995544, Training Accuracy= 0.93750\n",
+      "Iter 185600, Minibatch Loss= 2534.074463, Training Accuracy= 0.87500\n",
+      "Iter 192000, Minibatch Loss= 73.052612, Training Accuracy= 0.96875\n",
+      "Iter 198400, Minibatch Loss= 1609.606323, Training Accuracy= 0.93750\n",
+      "Iter 204800, Minibatch Loss= 1823.219727, Training Accuracy= 0.96875\n",
+      "Iter 211200, Minibatch Loss= 578.051086, Training Accuracy= 0.96875\n",
+      "Iter 217600, Minibatch Loss= 1532.326172, Training Accuracy= 0.89062\n",
+      "Iter 224000, Minibatch Loss= 769.775269, Training Accuracy= 0.95312\n",
+      "Iter 230400, Minibatch Loss= 2614.737793, Training Accuracy= 0.92188\n",
+      "Iter 236800, Minibatch Loss= 938.664368, Training Accuracy= 0.95312\n",
+      "Iter 243200, Minibatch Loss= 1520.495605, Training Accuracy= 0.93750\n",
+      "Iter 249600, Minibatch Loss= 657.419739, Training Accuracy= 0.95312\n",
+      "Iter 256000, Minibatch Loss= 522.802124, Training Accuracy= 0.90625\n",
+      "Iter 262400, Minibatch Loss= 211.188477, Training Accuracy= 0.96875\n",
+      "Iter 268800, Minibatch Loss= 520.451172, Training Accuracy= 0.92188\n",
+      "Iter 275200, Minibatch Loss= 1418.759155, Training Accuracy= 0.89062\n",
+      "Iter 281600, Minibatch Loss= 241.748596, Training Accuracy= 0.96875\n",
+      "Iter 288000, Minibatch Loss= 0.000000, Training Accuracy= 1.00000\n",
+      "Iter 294400, Minibatch Loss= 1535.772827, Training Accuracy= 0.92188\n",
+      "Optimization Finished!\n",
+      "Testing Accuracy: 0.980469\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Launch the graph\n",
+    "with tf.Session() as sess:\n",
+    "    sess.run(init)\n",
+    "    step = 1\n",
+    "    # Keep training until reach max iterations\n",
+    "    while step * batch_size < training_iters:\n",
+    "        batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
+    "        # Fit training using batch data\n",
+    "        sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys, keep_prob: dropout})\n",
+    "        if step % display_step == 0:\n",
+    "            # Calculate batch accuracy\n",
+    "            acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
+    "            # Calculate batch loss\n",
+    "            loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
+    "            print \"Iter \" + str(step*batch_size) + \", Minibatch Loss= \" \\\n",
+    "                  + \"{:.6f}\".format(loss) + \", Training Accuracy= \" + \"{:.5f}\".format(acc)\n",
+    "        step += 1\n",
+    "    print \"Optimization Finished!\"\n",
+    "    # Calculate accuracy for 256 mnist test images\n",
+    "    print \"Testing Accuracy:\", sess.run(accuracy, feed_dict={x: mnist.test.images[:256], \n",
+    "                                                             y: mnist.test.labels[:256], \n",
+    "                                                             keep_prob: 1.})"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "IPython (Python 2.7)",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

notebooks/3 - Neural Networks/convolutional_network.ipynb

@@ -0,0 +1,324 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# A Convolutional Networ implementation example using TensorFlow library.\n",
+    "# This example is using the MNIST database of handwritten digits\n",
+    "# (http://yann.lecun.com/exdb/mnist/)\n",
+    "\n",
+    "# Author: Aymeric Damien\n",
+    "# Project: https://github.com/aymericdamien/TensorFlow-Examples/"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Import MINST data\n",
+    "import input_data\n",
+    "mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import tensorflow as tf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Parameters\n",
+    "learning_rate = 0.001\n",
+    "training_iters = 100000\n",
+    "batch_size = 128\n",
+    "display_step = 20"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Network Parameters\n",
+    "n_input = 784 # MNIST data input (img shape: 28*28)\n",
+    "n_classes = 10 # MNIST total classes (0-9 digits)\n",
+    "dropout = 0.75 # Dropout, probability to keep units"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# tf Graph input\n",
+    "x = tf.placeholder(tf.types.float32, [None, n_input])\n",
+    "y = tf.placeholder(tf.types.float32, [None, n_classes])\n",
+    "keep_prob = tf.placeholder(tf.types.float32) #dropout (keep probability)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Create model\n",
+    "def conv2d(img, w, b):\n",
+    "    return tf.nn.relu(tf.nn.bias_add(tf.nn.conv2d(img, w, strides=[1, 1, 1, 1], \n",
+    "                                                  padding='SAME'),b))\n",
+    "\n",
+    "def max_pool(img, k):\n",
+    "    return tf.nn.max_pool(img, ksize=[1, k, k, 1], strides=[1, k, k, 1], padding='SAME')\n",
+    "\n",
+    "def conv_net(_X, _weights, _biases, _dropout):\n",
+    "    # Reshape input picture\n",
+    "    _X = tf.reshape(_X, shape=[-1, 28, 28, 1])\n",
+    "\n",
+    "    # Convolution Layer\n",
+    "    conv1 = conv2d(_X, _weights['wc1'], _biases['bc1'])\n",
+    "    # Max Pooling (down-sampling)\n",
+    "    conv1 = max_pool(conv1, k=2)\n",
+    "    # Apply Dropout\n",
+    "    conv1 = tf.nn.dropout(conv1, _dropout)\n",
+    "\n",
+    "    # Convolution Layer\n",
+    "    conv2 = conv2d(conv1, _weights['wc2'], _biases['bc2'])\n",
+    "    # Max Pooling (down-sampling)\n",
+    "    conv2 = max_pool(conv2, k=2)\n",
+    "    # Apply Dropout\n",
+    "    conv2 = tf.nn.dropout(conv2, _dropout)\n",
+    "\n",
+    "    # Fully connected layer\n",
+    "    # Reshape conv2 output to fit dense layer input\n",
+    "    dense1 = tf.reshape(conv2, [-1, _weights['wd1'].get_shape().as_list()[0]]) \n",
+    "    # Relu activation\n",
+    "    dense1 = tf.nn.relu(tf.add(tf.matmul(dense1, _weights['wd1']), _biases['bd1']))\n",
+    "    # Apply Dropout\n",
+    "    dense1 = tf.nn.dropout(dense1, _dropout) # Apply Dropout\n",
+    "\n",
+    "    # Output, class prediction\n",
+    "    out = tf.add(tf.matmul(dense1, _weights['out']), _biases['out'])\n",
+    "    return out"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Store layers weight & bias\n",
+    "weights = {\n",
+    "    # 5x5 conv, 1 input, 32 outputs\n",
+    "    'wc1': tf.Variable(tf.random_normal([5, 5, 1, 32])), \n",
+    "    # 5x5 conv, 32 inputs, 64 outputs\n",
+    "    'wc2': tf.Variable(tf.random_normal([5, 5, 32, 64])), \n",
+    "    # fully connected, 7*7*64 inputs, 1024 outputs\n",
+    "    'wd1': tf.Variable(tf.random_normal([7*7*64, 1024])), \n",
+    "    # 1024 inputs, 10 outputs (class prediction)\n",
+    "    'out': tf.Variable(tf.random_normal([1024, n_classes])) \n",
+    "}\n",
+    "\n",
+    "biases = {\n",
+    "    'bc1': tf.Variable(tf.random_normal([32])),\n",
+    "    'bc2': tf.Variable(tf.random_normal([64])),\n",
+    "    'bd1': tf.Variable(tf.random_normal([1024])),\n",
+    "    'out': tf.Variable(tf.random_normal([n_classes]))\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Construct model\n",
+    "pred = conv_net(x, weights, biases, keep_prob)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Define loss and optimizer\n",
+    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))\n",
+    "optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Evaluate model\n",
+    "correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))\n",
+    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.types.float32))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Initializing the variables\n",
+    "init = tf.initialize_all_variables()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Iter 2560, Minibatch Loss= 26046.011719, Training Accuracy= 0.21094\n",
+      "Iter 5120, Minibatch Loss= 10456.769531, Training Accuracy= 0.52344\n",
+      "Iter 7680, Minibatch Loss= 6273.207520, Training Accuracy= 0.71875\n",
+      "Iter 10240, Minibatch Loss= 6276.231445, Training Accuracy= 0.64062\n",
+      "Iter 12800, Minibatch Loss= 4188.221680, Training Accuracy= 0.77344\n",
+      "Iter 15360, Minibatch Loss= 2717.077637, Training Accuracy= 0.80469\n",
+      "Iter 17920, Minibatch Loss= 4057.120361, Training Accuracy= 0.81250\n",
+      "Iter 20480, Minibatch Loss= 1696.550415, Training Accuracy= 0.87500\n",
+      "Iter 23040, Minibatch Loss= 2525.317627, Training Accuracy= 0.85938\n",
+      "Iter 25600, Minibatch Loss= 2341.906738, Training Accuracy= 0.87500\n",
+      "Iter 28160, Minibatch Loss= 4200.535156, Training Accuracy= 0.79688\n",
+      "Iter 30720, Minibatch Loss= 1888.964355, Training Accuracy= 0.89062\n",
+      "Iter 33280, Minibatch Loss= 2167.645996, Training Accuracy= 0.84375\n",
+      "Iter 35840, Minibatch Loss= 1932.107544, Training Accuracy= 0.89844\n",
+      "Iter 38400, Minibatch Loss= 1562.430054, Training Accuracy= 0.90625\n",
+      "Iter 40960, Minibatch Loss= 1676.755249, Training Accuracy= 0.84375\n",
+      "Iter 43520, Minibatch Loss= 1003.626099, Training Accuracy= 0.93750\n",
+      "Iter 46080, Minibatch Loss= 1176.615479, Training Accuracy= 0.86719\n",
+      "Iter 48640, Minibatch Loss= 1260.592651, Training Accuracy= 0.88281\n",
+      "Iter 51200, Minibatch Loss= 1399.667969, Training Accuracy= 0.86719\n",
+      "Iter 53760, Minibatch Loss= 1259.961426, Training Accuracy= 0.89844\n",
+      "Iter 56320, Minibatch Loss= 1415.800781, Training Accuracy= 0.89062\n",
+      "Iter 58880, Minibatch Loss= 1835.365967, Training Accuracy= 0.85156\n",
+      "Iter 61440, Minibatch Loss= 1395.168823, Training Accuracy= 0.90625\n",
+      "Iter 64000, Minibatch Loss= 973.283569, Training Accuracy= 0.88281\n",
+      "Iter 66560, Minibatch Loss= 818.093811, Training Accuracy= 0.92969\n",
+      "Iter 69120, Minibatch Loss= 1178.744263, Training Accuracy= 0.92188\n",
+      "Iter 71680, Minibatch Loss= 845.889709, Training Accuracy= 0.89844\n",
+      "Iter 74240, Minibatch Loss= 1259.505615, Training Accuracy= 0.90625\n",
+      "Iter 76800, Minibatch Loss= 738.037109, Training Accuracy= 0.89844\n",
+      "Iter 79360, Minibatch Loss= 862.499146, Training Accuracy= 0.93750\n",
+      "Iter 81920, Minibatch Loss= 739.704041, Training Accuracy= 0.90625\n",
+      "Iter 84480, Minibatch Loss= 652.880310, Training Accuracy= 0.95312\n",
+      "Iter 87040, Minibatch Loss= 635.464600, Training Accuracy= 0.92969\n",
+      "Iter 89600, Minibatch Loss= 933.166626, Training Accuracy= 0.90625\n",
+      "Iter 92160, Minibatch Loss= 213.874893, Training Accuracy= 0.96094\n",
+      "Iter 94720, Minibatch Loss= 609.575684, Training Accuracy= 0.91406\n",
+      "Iter 97280, Minibatch Loss= 560.208008, Training Accuracy= 0.93750\n",
+      "Iter 99840, Minibatch Loss= 963.577148, Training Accuracy= 0.90625\n",
+      "Optimization Finished!\n",
+      "Testing Accuracy: 0.960938\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Launch the graph\n",
+    "with tf.Session() as sess:\n",
+    "    sess.run(init)\n",
+    "    step = 1\n",
+    "    # Keep training until reach max iterations\n",
+    "    while step * batch_size < training_iters:\n",
+    "        batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
+    "        # Fit training using batch data\n",
+    "        sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys, keep_prob: dropout})\n",
+    "        if step % display_step == 0:\n",
+    "            # Calculate batch accuracy\n",
+    "            acc = sess.run(accuracy, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
+    "            # Calculate batch loss\n",
+    "            loss = sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 1.})\n",
+    "            print \"Iter \" + str(step*batch_size) + \", Minibatch Loss= \" + \\\n",
+    "                  \"{:.6f}\".format(loss) + \", Training Accuracy= \" + \"{:.5f}\".format(acc)\n",
+    "        step += 1\n",
+    "    print \"Optimization Finished!\"\n",
+    "    # Calculate accuracy for 256 mnist test images\n",
+    "    print \"Testing Accuracy:\", sess.run(accuracy, feed_dict={x: mnist.test.images[:256], \n",
+    "                                                             y: mnist.test.labels[:256], \n",
+    "                                                             keep_prob: 1.})"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "IPython (Python 2.7)",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

notebooks/3 - Neural Networks/multilayer_perceptron.ipynb

@@ -0,0 +1,256 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# A Multilayer Perceptron implementation example using TensorFlow library.\n",
+    "# This example is using the MNIST database of handwritten digits\n",
+    "# (http://yann.lecun.com/exdb/mnist/)\n",
+    "\n",
+    "# Author: Aymeric Damien\n",
+    "# Project: https://github.com/aymericdamien/TensorFlow-Examples/"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Extracting /tmp/data/train-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/train-labels-idx1-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-images-idx3-ubyte.gz\n",
+      "Extracting /tmp/data/t10k-labels-idx1-ubyte.gz\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Import MINST data\n",
+    "import input_data\n",
+    "mnist = input_data.read_data_sets(\"/tmp/data/\", one_hot=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import tensorflow as tf"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Parameters\n",
+    "learning_rate = 0.001\n",
+    "training_epochs = 15\n",
+    "batch_size = 100\n",
+    "display_step = 1"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Network Parameters\n",
+    "n_hidden_1 = 256 # 1st layer num features\n",
+    "n_hidden_2 = 256 # 2nd layer num features\n",
+    "n_input = 784 # MNIST data input (img shape: 28*28)\n",
+    "n_classes = 10 # MNIST total classes (0-9 digits)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# tf Graph input\n",
+    "x = tf.placeholder(\"float\", [None, n_input])\n",
+    "y = tf.placeholder(\"float\", [None, n_classes])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Create model\n",
+    "def multilayer_perceptron(_X, _weights, _biases):\n",
+    "    #Hidden layer with RELU activation\n",
+    "    layer_1 = tf.nn.relu(tf.add(tf.matmul(_X, _weights['h1']), _biases['b1'])) \n",
+    "    #Hidden layer with RELU activation\n",
+    "    layer_2 = tf.nn.relu(tf.add(tf.matmul(layer_1, _weights['h2']), _biases['b2'])) \n",
+    "    return tf.matmul(layer_2, weights['out']) + biases['out']"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Store layers weight & bias\n",
+    "weights = {\n",
+    "    'h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),\n",
+    "    'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),\n",
+    "    'out': tf.Variable(tf.random_normal([n_hidden_2, n_classes]))\n",
+    "}\n",
+    "biases = {\n",
+    "    'b1': tf.Variable(tf.random_normal([n_hidden_1])),\n",
+    "    'b2': tf.Variable(tf.random_normal([n_hidden_2])),\n",
+    "    'out': tf.Variable(tf.random_normal([n_classes]))\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Construct model\n",
+    "pred = multilayer_perceptron(x, weights, biases)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Define loss and optimizer\n",
+    "# Softmax loss\n",
+    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) \n",
+    "# Adam Optimizer\n",
+    "optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost) "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Initializing the variables\n",
+    "init = tf.initialize_all_variables()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch: 0001 cost= 160.113980416\n",
+      "Epoch: 0002 cost= 38.665780694\n",
+      "Epoch: 0003 cost= 24.118004577\n",
+      "Epoch: 0004 cost= 16.440921303\n",
+      "Epoch: 0005 cost= 11.689460141\n",
+      "Epoch: 0006 cost= 8.469423468\n",
+      "Epoch: 0007 cost= 6.223237230\n",
+      "Epoch: 0008 cost= 4.560174118\n",
+      "Epoch: 0009 cost= 3.250516910\n",
+      "Epoch: 0010 cost= 2.359658795\n",
+      "Epoch: 0011 cost= 1.694081847\n",
+      "Epoch: 0012 cost= 1.167997509\n",
+      "Epoch: 0013 cost= 0.872986831\n",
+      "Epoch: 0014 cost= 0.630616366\n",
+      "Epoch: 0015 cost= 0.487381571\n",
+      "Optimization Finished!\n",
+      "Accuracy: 0.9462\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Launch the graph\n",
+    "with tf.Session() as sess:\n",
+    "    sess.run(init)\n",
+    "\n",
+    "    # Training cycle\n",
+    "    for epoch in range(training_epochs):\n",
+    "        avg_cost = 0.\n",
+    "        total_batch = int(mnist.train.num_examples/batch_size)\n",
+    "        # Loop over all batches\n",
+    "        for i in range(total_batch):\n",
+    "            batch_xs, batch_ys = mnist.train.next_batch(batch_size)\n",
+    "            # Fit training using batch data\n",
+    "            sess.run(optimizer, feed_dict={x: batch_xs, y: batch_ys})\n",
+    "            # Compute average loss\n",
+    "            avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys})/total_batch\n",
+    "        # Display logs per epoch step\n",
+    "        if epoch % display_step == 0:\n",
+    "            print \"Epoch:\", '%04d' % (epoch+1), \"cost=\", \"{:.9f}\".format(avg_cost)\n",
+    "\n",
+    "    print \"Optimization Finished!\"\n",
+    "\n",
+    "    # Test model\n",
+    "    correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))\n",
+    "    # Calculate accuracy\n",
+    "    accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\n",
+    "    print \"Accuracy:\", accuracy.eval({x: mnist.test.images, y: mnist.test.labels})"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "IPython (Python 2.7)",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}

notebooks/4 - Multi GPU/multigpu_basics.ipynb

@@ -0,0 +1,178 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Basic Multi GPU computation example using TensorFlow library.\n",
+    "\n",
+    "# Author: Aymeric Damien\n",
+    "# Project: https://github.com/aymericdamien/TensorFlow-Examples/\n",
+    "\n",
+    "# This tutorial requires your machine to have 2 GPUs\n",
+    "# \"/cpu:0\": The CPU of your machine.\n",
+    "# \"/gpu:0\": The first GPU of your machine\n",
+    "# \"/gpu:1\": The second GPU of your machine\n",
+    "# For this example, we are using 2 GTX-980"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import tensorflow as tf\n",
+    "import datetime"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "#Processing Units logs\n",
+    "log_device_placement = True\n",
+    "\n",
+    "#num of multiplications to perform\n",
+    "n = 10"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "# Example: compute A^n + B^n on 2 GPUs\n",
+    "\n",
+    "# Create random large matrix\n",
+    "A = np.random.rand(1e4, 1e4).astype('float32')\n",
+    "B = np.random.rand(1e4, 1e4).astype('float32')\n",
+    "\n",
+    "# Creates a graph to store results\n",
+    "c1 = []\n",
+    "c2 = []\n",
+    "\n",
+    "# Define matrix power\n",
+    "def matpow(M, n):\n",
+    "    if n < 1: #Abstract cases where n < 1\n",
+    "        return M\n",
+    "    else:\n",
+    "        return tf.matmul(M, matpow(M, n-1))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Single GPU computing\n",
+    "\n",
+    "with tf.device('/gpu:0'):\n",
+    "    a = tf.constant(A)\n",
+    "    b = tf.constant(B)\n",
+    "    #compute A^n and B^n and store results in c1\n",
+    "    c1.append(matpow(a, n))\n",
+    "    c1.append(matpow(b, n))\n",
+    "\n",
+    "with tf.device('/cpu:0'):\n",
+    "  sum = tf.add_n(c1) #Addition of all elements in c1, i.e. A^n + B^n\n",
+    "\n",
+    "t1_1 = datetime.datetime.now()\n",
+    "with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:\n",
+    "    # Runs the op.\n",
+    "    sess.run(sum)\n",
+    "t2_1 = datetime.datetime.now()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": [
+    "# Multi GPU computing\n",
+    "# GPU:0 computes A^n\n",
+    "with tf.device('/gpu:0'):\n",
+    "    #compute A^n and store result in c2\n",
+    "    a = tf.constant(A)\n",
+    "    c2.append(matpow(a, n))\n",
+    "\n",
+    "#GPU:1 computes B^n\n",
+    "with tf.device('/gpu:1'):\n",
+    "    #compute B^n and store result in c2\n",
+    "    b = tf.constant(B)\n",
+    "    c2.append(matpow(b, n))\n",
+    "\n",
+    "with tf.device('/cpu:0'):\n",
+    "  sum = tf.add_n(c2) #Addition of all elements in c2, i.e. A^n + B^n\n",
+    "\n",
+    "t1_2 = datetime.datetime.now()\n",
+    "with tf.Session(config=tf.ConfigProto(log_device_placement=log_device_placement)) as sess:\n",
+    "    # Runs the op.\n",
+    "    sess.run(sum)\n",
+    "t2_2 = datetime.datetime.now()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Single GPU computation time: 0:00:11.833497\n",
+      "Multi GPU computation time: 0:00:07.085913\n"
+     ]
+    }
+   ],
+   "source": [
+    "print \"Single GPU computation time: \" + str(t2_1-t1_1)\n",
+    "print \"Multi GPU computation time: \" + str(t2_2-t1_2)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 2",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}