Tek Shinobi

We will analyze the famous IRIS dataset.
Before we start, here are the basic steps that any typical Machine Learning based Data Analysis workflow consists of. (Note: if text inside figure appears small, please increase the font size temporarily by Ctrl+roll-mouse-scroller)

Most of our time will be spent in Phases 1 and 2.

Source code is hosted on github here

Lets walk the process with IRIS dataset.

Phase 1 : Data Preparation
Since IRIS dataset comes prepackaged with sklean, we save the trouble of downloading the dataset. Furthermore, the dataset is already cleaned and labeled. So we just need to put the data in a format we will use in the application.

First, let me dump all the includes. These will be used at various times during the coding.

from sklearn import datasets
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import Dense
from sklearn.preprocessing import LabelBinarizer
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_val_score

Next we begin the data preparation phase.
Since the IRIS dataset involves classification of flowers into three kinds: setosa, versicolor and virginica,
it behooves us to use one hot encoding to encode the target. The dataset uses 0,1 and 2 for respective classes. We will convert these into one-hot encoded vectors.
We will use the value of “seed” later in random_state

Step 1: Create Dataframes for features and target

encoder = LabelBinarizer()
seed = 42

iris = datasets.load_iris()
iris_data_df = pd.DataFrame(data=iris.data, columns=iris.feature_names,
                       dtype=np.float32)
target = encoder.fit_transform(iris.target)
iris_target_df = pd.DataFrame(data=target, columns=iris.target_names) 

Step 2: Create training and testing datasets

X_train,X_test,y_train,y_test = train_test_split(iris_data_df,
                                                 iris_target_df,
                                                 test_size=0.30,
                                                 random_state=seed)

Step 3: Feature scaling.

ML algos perform best when all dataset features have the same scale. Particularly, in Neural Nets, we use MinMaxScaler with range between 0 and 1.
Also, since MinMaxScalar transformation makes us lose the column and index labels, we need to recreate the dataframes for scaled data and target.

scaler = MinMaxScaler(feature_range=(0,1))

X_train = pd.DataFrame(scaler.fit_transform(X_train),
                               columns=X_train.columns,
                               index=X_train.index)
X_test = pd.DataFrame(scaler.transform(X_test),
                           columns=X_test.columns,
                           index=X_test.index)

Phase 2 : Model Building and Model Cross Validation

In this phase we build various models and use cross validation to see how they perform.

Since we wish to keep things simple, we will be building just one model, that of DNN or Dense Neural Network.

Step 1: Build model

Since we are building a classifier (to classify whether a flower is setosa, versicolor or virginia), we will use a softmax activation function in the output.

def model():
    """build the Keras model callback"""
    model = Sequential()
    model.add(Dense(8, input_dim=4, activation='tanh', name='layer_1'))
    model.add(Dense(10, activation='tanh', name='layer_2'))
    model.add(Dense(10, activation='tanh', name='layer_3'))
    model.add(Dense(3, activation='softmax', name='output_layer'))
    
    model.compile(loss="categorical_crossentropy",
                  optimizer="adam",
                  metrics=['accuracy'])
    return model

Step 2: Create estimator

estimator = KerasClassifier(
        build_fn=model,
        epochs=200, batch_size=20,
        verbose=2)

Step 3: Do the Model cross validation

Since the dataset is quite small, we can get away with just 5 fold cross validation.
On bigger datasets, increase the folds by modifying n_splits

kfold = KFold(n_splits=5, shuffle=True, random_state=seed)
results = cross_val_score(estimator, X_train, y_train, cv=kfold)
print("Model Performance: mean: %.2f%% std: (%.2f%%)" % (results.mean()*100, results.std()*100))

At this stage, we can run the code. The whole code up to this point looks like this:

from sklearn import datasets
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import Dense
from sklearn.preprocessing import LabelBinarizer
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_val_score

encoder = LabelBinarizer()
seed = 42

iris = datasets.load_iris()
iris_data_df = pd.DataFrame(data=iris.data, columns=iris.feature_names,
                       dtype=np.float32)
target = encoder.fit_transform(iris.target)
iris_target_df = pd.DataFrame(data=target, columns=iris.target_names)

X_train,X_test,y_train,y_test = train_test_split(iris_data_df,
                                                 iris_target_df,
                                                 test_size=0.30,
                                                 random_state=seed)

scaler = MinMaxScaler(feature_range=(0,1))

X_train = pd.DataFrame(scaler.fit_transform(X_train),
                               columns=X_train.columns,
                               index=X_train.index)
X_test = pd.DataFrame(scaler.transform(X_test),
                           columns=X_test.columns,
                           index=X_test.index)

def model():
    """build the Keras model callback"""
    model = Sequential()
    model.add(Dense(8, input_dim=4, activation='tanh', name='layer_1'))
    model.add(Dense(10, activation='tanh', name='layer_2'))
    model.add(Dense(10, activation='tanh', name='layer_3'))
    model.add(Dense(3, activation='softmax', name='output_layer'))
    
    model.compile(loss="categorical_crossentropy",
                  optimizer="adam",
                  metrics=['accuracy'])
    return model

estimator = KerasClassifier(
        build_fn=model,
        epochs=200, batch_size=20,
        verbose=2)
kfold = KFold(n_splits=5, shuffle=True, random_state=seed)
results = cross_val_score(estimator, X_train, y_train, cv=kfold)
print("Model Performance: mean: %.2f%% std: (%.2f%%)" % (results.mean()*100, results.std()*100))

The mean performance is around 93% which is quite good.

Phase 3 : Model Selection
In our case this is very easy. We have just one model. DNN model we created in Phase 2. The mean performance is around 93% which is quite good.

If we created multiple models in Phase 2, we will compare their performances in this phase and choose the right model.

Phase 4 : Model Selection
Now we are in the fast lane. This phase is usually pretty quick. All we have to do is to fit the model again.

Here we instantiate the model and fit it.

model = model()
model.fit(
       X_train,
       y_train,
       epochs=200,
       shuffle=True, # shuffle data randomly.
       #NNs perform best on randomly shuffled data
       verbose=2 # this will tell keras to print more detailed info
       # during trainnig to know what is going on
       )

Then we do model evaluation. This is the place where we will use the test data for the first time. This is our final check of the model before we deploy the model in production.

#run the test dataset
test_error_rate = model.evaluate(X_test, y_test, verbose=0)
print(
      "{} : {:.2f}%".format(model.metrics_names[1],
              test_error_rate[1]*100))
print(
      "{} : {:.2f}%".format(model.metrics_names[0],
              test_error_rate[0]*100))

You can run the code again to see how the model performs on the test data. The test_error_rate gives us an idea of how well we did on the test data.
We get around 98% accuracy (categorical_accuracy) which is pretty good.

OK. So now its time to try the model out to predict some unseen data. Since we didn’t train the model on X_test, we will use the same data to predict. We will also write a helper function for evaluating the results.

predicted_targets = model.predict_classes(X_test)
true_targets = encoder.inverse_transform(y_test.values)

def performance_tracker(actual, expected):
    flowers = {0:'setosa', 1:'versicolor', 2:'virginica'}
    print("Flowers in test set: Setosa={} Versicolor={} Virginica={}".format(
            y_test.setosa.sum(), y_test.versicolor.sum(),
            y_test.virginica.sum()))
    for act,exp in zip(actual, expected):
        if act != exp:
            print("ERROR: {} predicted as {}".format(flowers[exp],
                  flowers[act]))
            
performance_tracker(predicted_targets, true_targets)

Cool, there were 45 flowers in the test set and only one sample was mis-predicted. Rest were OK.