A Comprehensive Machine Learning Code for Dermal Image Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This project deals with performance evaluation of machine learning techniques in classifiying malignant, begign melanoma and normal mole.Skin tumour is uncontrolled growth of cells in the skin which may be cancerous. Melanoma is one of the common types of skin cancer. The aim is to develop computer aided diagnosis for skin tumours. The dermal images of three types such as benign tumour, malignant melanoma and normal moles obtained from the authorised PH2 database and other sources. The details are provided in the attached README file. Pre-processing is performed to remove hair cells. Contour based level set technique for segmentation of the lesion is performed from which clinical, texture and morphological features are extracted. The significant features are obtained using Random Subset Feature Selection technique and classification is carried out using machine learning algorithms. To begin with, we load the libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from matplotlib import pyplot\n",
    "from pandas import read_csv\n",
    "from pandas import set_option\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.ensemble import ExtraTreesClassifier\n",
    "import seaborn as sns\n",
    "sns.set(color_codes=True)\n",
    "%matplotlib inline "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset is loaded in the form of a .csv file containing 15 attributes and 1500 instances and the 16th attribute is the class, depicting '1' for malignant, '2' for begign and '3'for normal mole."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.getcwd()\n",
    "filename = 'Book1.csv'\n",
    "names = [\"area\", 'asymmetry_index', 'GD', 'SD', 'perimeter', 'circularity_index', \n",
    "         'compact_index', 'entropy', 'color_mean', 'color_std','contrast',\n",
    "         'correlation', 'energy', 'homogenity', 'mean', 'class']\n",
    "data = read_csv(filename, names=names)\n",
    "print(data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a look at the data types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_option('display.max_rows', 500)\n",
    "print(data.dtypes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a peek at the first 20 rows of the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_option('display.width', 100)\n",
    "print(data.head(20))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Its time to describe the data, lets change the prediction to 3 decimal points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_option('precision', 3)\n",
    "print(data.describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a quick look at the breakdown of the class values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(data.groupby('class').size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets measure the correlation between the attributes. Correlation refers to the relationship between two variables and how they may or may not change together. The most common method for calculating correlation is Pearson's Correlation Coefficient, assuming a normal distribution of the attributes involved. A correlation of -1 or 1 shows a full negative or positive correlation respectively. Whereas a value of 0 shows no correlation at all. Some machine learning algorithms like linear and logistic regression can show poor performance if there are highly correlated attributes in your dataset. As such, it is a good\n",
    "idea to review all of the pairwise correlations of the attributes in your dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_option('display.width', 100)\n",
    "correlations = data.corr(method='pearson')\n",
    "print(correlations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Skew refers to a distribution that is assumed Gaussian (normal or bell curve) that is shifted or squashed in one direction or another. Many machine learning algorithms assume a Gaussian distribution. Knowing that an attribute has a skew may allow you to perform data preparation to correct the skew and later improve the accuracy of your models. You can calculate the skew of each attribute using the skew() function on the Pandas DataFrame.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "skew = data.skew()\n",
    "print(skew)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There appears to be quite a lot of skew in the data, needed to be corrected by box-cox transform or other methods. However, ensemble methods are less sensitive to data distribution. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unimodal data vislualizations: visualizations of individual attributes: \n",
    "It is often useful to look at your data using multiple different visualizations in order to spark ideas. Let's look at histograms of each attribute to get a sense of the data distributions. A fast way to get an idea of the distribution of each attribute is to look at histograms. Histograms group data into bins and provide you a count of the number of observations in each bin. From the shape of the bins you can quickly get a feeling for whether an attribute is Gaussian, skewed\n",
    "or even has an exponential distribution. It can also help you see possible outliers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pyplot.figure(figsize=(20,10), dpi=100)\n",
    "data.hist(sharex = False, sharey = False, xlabelsize = 1, ylabelsize = 1)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that most of the features have a skewed distribution. Some sort of data standardization and scaling becomes necessary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Density plot: \n",
    "Lets look at the same perspective of the data using density plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.plot(kind='density', subplots=True, layout=(4,4), sharex=False, legend=False,\n",
    "fontsize=1)\n",
    "pyplot.figure(figsize=(20,10), dpi=100)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets visualize the box plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.plot(kind='box', subplots=True, layout=(4,4), sharex=False, sharey=False)\n",
    "pyplot.figure(figsize=(20,10), dpi=100)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the spread of attributes is quite different. Some features like contrast, circularity index, and compact-index appear quite skewed towards smaller values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now perform multimodal data visualizations by plotting Multivariate Plots that shows the interactions between multiple variables in the dataset. This can be done with Correlation Matrix Plot and Scatter Plot Matrix. Correlation gives an indication of how related the changes are between two variables. If two variables change in the same direction they are positively correlated. If they change in opposite directions together (one goes up, one goes down), then they are negatively correlated. You can calculate the correlation between each pair of attributes. This is called a correlation matrix. You can then plot the correlation matrix and get an idea of which variables have a high correlation with each other. This is useful to know, because some machine learning algorithms like linear and logistic regression can have poor performance if there are highly correlated input variables in your data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# correlation matrix: visualizing the correlation between the attributes\n",
    "correlations = data.corr()\n",
    "# plot correlation matrix\n",
    "fig = pyplot.figure(figsize=(20,10), dpi=100)\n",
    "ax = fig.add_subplot(111)\n",
    "cax = ax.matshow(correlations, vmin=-1, vmax=1)\n",
    "fig.colorbar(cax)\n",
    "ticks = numpy.arange(0,15,1)\n",
    "ax.set_xticks(ticks)\n",
    "ax.set_yticks(ticks)\n",
    "ax.set_xticklabels(names)\n",
    "ax.set_yticklabels(names)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the matrix is symmetrical, i.e. the bottom left of the matrix is the same as the top right. This is useful as we can see two different views on the same data in one plot. We can also see that each variable is perfectly positively correlated with each other (as you would have expected) in the diagonal line from top left to bottom right. Patches of white shows strong negative correlation among the variables. We will now plot a Scatter Plot Matrix. A scatter plot shows the relationship between two variables as dots in two dimensions, one axis for each attribute. You can create a scatter plot for each pair of attributes in your data. Drawing all these scatter plots together is called a scatter plot matrix. Scatter plots are useful for spotting structured relationships between variables, like whether you could summarize the relationship between two variables with a line. Attributes with structured relationships may also be correlated and good candidates for removal from your dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#you can use seaborn that gives a clear picture\n",
    "sns.pairplot(data); #plotting paired visualizations, equivalent to scatter matrix\n",
    "pyplot.figure(figsize=(20,10), dpi=100)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot shows some sort of linear correlation between the variables. A few appear to be linearly separable. Lets begin the evaluation. Lets evaluate the performance of different models including Logistic Regression, Linear Discriminant Analysis, KNeighborsClassifier, Decision Trees, and SVM. We will perform 10-fold cross-validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Split-out validation dataset\n",
    "array = data.values\n",
    "X = array[:,0:15].astype(float)\n",
    "Y = array[:,15].astype(int)\n",
    "validation_size = 0.20\n",
    "seed = 7\n",
    "X_train, X_validation, Y_train, Y_validation = train_test_split(X, Y,\n",
    "test_size=validation_size, random_state=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test options and evaluation metric\n",
    "num_folds = 10\n",
    "seed = 7\n",
    "scoring = 'accuracy'\n",
    "#creating a baseline of performance for this problem\n",
    "# Spot-Check Algorithms: two linear and four non-linear\n",
    "models = []\n",
    "models.append(('LR', LogisticRegression()))\n",
    "models.append(('LDA', LinearDiscriminantAnalysis()))\n",
    "models.append(('KNN', KNeighborsClassifier()))\n",
    "models.append(('CART', DecisionTreeClassifier()))\n",
    "models.append(('NB', GaussianNB()))\n",
    "models.append(('SVM', SVC()))\n",
    "results = []\n",
    "names = []\n",
    "for name, model in models:\n",
    "    kfold = KFold(n_splits=10, random_state=7)\n",
    "    cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)\n",
    "    results.append(cv_results)\n",
    "    names.append(name)\n",
    "    msg = \"%s: %f (%f)\" % (name, cv_results.mean()*100, cv_results.std()*100)\n",
    "    print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets compare the performance of the algorithms by measuring the distribution of scores across all cross-validation folds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = pyplot.figure(figsize=(20,10), dpi=100)\n",
    "fig.suptitle('Algorithm Comparison')\n",
    "ax = fig.add_subplot(111)\n",
    "bp = pyplot.boxplot(results, patch_artist = True)\n",
    "## change outline color, fill color and linewidth of the boxes\n",
    "for box in bp['boxes']:\n",
    "    box.set(color='Green')\n",
    "    box.set(facecolor='white')\n",
    "for whisker in bp['whiskers']:\n",
    "    whisker.set(color=\"Black\")\n",
    "for cap in bp['caps']:\n",
    "    cap.set(color=\"Gray\")\n",
    "for median in bp['medians']:\n",
    "    median.set(color=\"red\")\n",
    "# change the style of fliers and their fill\n",
    "for flier in bp['fliers']:\n",
    "    flier.set(marker='o', color='#e7298a', alpha=0.3)\n",
    "ax.set_xticklabels(names)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets evaluate our algorithms with standardized data. We suspect that the differing distributions of the raw data may be negatively impacting the skill of some of the algorithms. Let's evaluate the same algorithms with a standardized copy of the #dataset where the data is transformed such that each attribute has a mean value of zero and a standard deviation of one. We also need to avoid data leakage when we transform the data. A good way to avoid leakage is to use pipelines that standardize the data and build the model for each fold in the cross-validation test harness. That way we can get a fair estimation of how each model with standardized data might perform on unseen data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Standardize the dataset\n",
    "pipelines = []\n",
    "pipelines.append(('ScaledLR', Pipeline([('Scaler', StandardScaler()), ('LR',\n",
    "LogisticRegression())])))\n",
    "pipelines.append(('ScaledLDA', Pipeline([('Scaler', StandardScaler()),('LDA',\n",
    "LinearDiscriminantAnalysis())])))\n",
    "pipelines.append(('ScaledKNN', Pipeline([('Scaler', StandardScaler()),('KNN',\n",
    "KNeighborsClassifier())])))\n",
    "pipelines.append(('ScaledCART', Pipeline([('Scaler', StandardScaler()),('CART',\n",
    "DecisionTreeClassifier())])))\n",
    "pipelines.append(('ScaledNB', Pipeline([('Scaler', StandardScaler()),('NB',\n",
    "GaussianNB())])))\n",
    "pipelines.append(('ScaledSVM', Pipeline([('Scaler', StandardScaler()),('SVM', SVC())])))\n",
    "results = []\n",
    "names = []\n",
    "for name, model in pipelines:\n",
    "    kfold = KFold(n_splits=num_folds, random_state=seed)\n",
    "    cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)\n",
    "    results.append(cv_results)\n",
    "    names.append(name)\n",
    "    msg = \"%s: %f (%f)\" % (name, cv_results.mean()*100, cv_results.std()*100)\n",
    "    print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets compare the performance of Scaled Algorithms and measure the distribution of scores across all cross-validation folds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = pyplot.figure(figsize=(20,10), dpi=100)\n",
    "fig.suptitle('Scaled Algorithms Comparison')\n",
    "ax = fig.add_subplot(111)\n",
    "bp = pyplot.boxplot(results, patch_artist = True)\n",
    "## change outline color, fill color and linewidth of the boxes\n",
    "for box in bp['boxes']:\n",
    "    box.set(color='Green')\n",
    "    box.set(facecolor='white')\n",
    "for whisker in bp['whiskers']:\n",
    "    whisker.set(color=\"Black\")\n",
    "for cap in bp['caps']:\n",
    "    cap.set(color=\"Gray\")\n",
    "for median in bp['medians']:\n",
    "    median.set(color=\"red\")\n",
    "# change the style of fliers and their fill\n",
    "for flier in bp['fliers']:\n",
    "    flier.set(marker='o', color='#e7298a', alpha=0.3)\n",
    "ax.set_xticklabels(names)\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results suggest digging deeper into the CART abd SVM algorithms. It is very likely that configuration beyond the default may yield even more accurate models. But before that, we can try our hands with ensemble methods as well as they are known to produce better results. Lets see how they work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ensembles = []\n",
    "ensembles.append(('AB', AdaBoostClassifier()))\n",
    "ensembles.append(('GBM', GradientBoostingClassifier()))\n",
    "ensembles.append(('RF', RandomForestClassifier()))\n",
    "ensembles.append(('ET', ExtraTreesClassifier()))\n",
    "results = []\n",
    "names = []\n",
    "for name, model in ensembles:\n",
    "    kfold = KFold(n_splits=num_folds, random_state=seed)\n",
    "    cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)\n",
    "    results.append(cv_results)\n",
    "    names.append(name)\n",
    "    msg = \"%s: %f (%f)\" % (name, cv_results.mean(), cv_results.std())\n",
    "    print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, both random forest classifier and extra trees classifier are doing great with the data in comparison to CART and SVM. Lets try with the standardized data to look for the improvement in performance. Lets use pipelines again for our task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Standardized ensembles\n",
    "ensembles = []\n",
    "ensembles.append(('ScaledAB', Pipeline([('Scaler', StandardScaler()),('AB',\n",
    "AdaBoostClassifier())])))\n",
    "ensembles.append(('ScaledGBM', Pipeline([('Scaler', StandardScaler()),('GBM',\n",
    "GradientBoostingClassifier())])))\n",
    "ensembles.append(('ScaledRF', Pipeline([('Scaler', StandardScaler()),('RF',\n",
    "RandomForestClassifier())])))\n",
    "ensembles.append(('ScaledET', Pipeline([('Scaler', StandardScaler()),('ET',\n",
    "ExtraTreesClassifier())])))\n",
    "results = []\n",
    "names = []\n",
    "for name, model in ensembles:\n",
    "    kfold = KFold(n_splits=num_folds, random_state=seed)\n",
    "    cv_results = cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)\n",
    "    results.append(cv_results)\n",
    "    names.append(name)\n",
    "    msg = \"%s: %f (%f)\" % (name, cv_results.mean(), cv_results.std())\n",
    "    print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the standardized data, the extra trees classifier performs marginally better than the random forest classifier. Lets now tune our ensembles. The default number of bagging methods to perform (n estimators) is 100. This is a good candidate parameter for the bagging classifiers to tune. Often, the larger the number of bagging stages, the better the performance but the longer the training time. Below we define a parameter grid n estimators values from 50 to 400 in increments of 50. Each setting is evaluated using 10-fold cross-validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tune scaled Random Forest Classifier\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "param_grid = dict(n_estimators=numpy.array([50,100,150,200,250,300,350,400]))\n",
    "model = RandomForestClassifier(random_state=seed)\n",
    "kfold = KFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))\n",
    "means = grid_result.cv_results_['mean_test_score']\n",
    "stds = grid_result.cv_results_['std_test_score']\n",
    "params = grid_result.cv_results_['params']\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tune scaled Extra Trees Classifier\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "param_grid = dict(n_estimators=numpy.array([50,100,150,200,250,300,350,400]))\n",
    "model = ExtraTreesClassifier(random_state=seed)\n",
    "kfold = KFold(n_splits=num_folds, random_state=seed)\n",
    "grid = GridSearchCV(estimator=model, param_grid=param_grid, scoring=scoring, cv=kfold)\n",
    "grid_result = grid.fit(rescaledX, Y_train)\n",
    "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))\n",
    "means = grid_result.cv_results_['mean_test_score']\n",
    "stds = grid_result.cv_results_['std_test_score']\n",
    "params = grid_result.cv_results_['params']\n",
    "for mean, stdev, param in zip(means, stds, params):\n",
    "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We observe that the extra Trees Classifier performing marginally better than that Random Forest Classifier. We will use it as the final model to predict on the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Finalize the model\n",
    "scaler = StandardScaler().fit(X_train)\n",
    "rescaledX = scaler.transform(X_train)\n",
    "model = ExtraTreesClassifier(random_state=seed, n_estimators=150)\n",
    "model.fit(rescaledX, Y_train)\n",
    "# estimate accuracy on validation dataset\n",
    "rescaledValidationX = scaler.transform(X_validation)\n",
    "predictions = model.predict(rescaledValidationX)\n",
    "print(accuracy_score(Y_validation, predictions))\n",
    "print(confusion_matrix(Y_validation, predictions))\n",
    "print(classification_report(Y_validation, predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, extra trees classifier perform on par with the test data, giving around 99% precision, recall, and F1-score, serving to be an excellent candidate to classify DERMAL images under study. The efficiency of the algorithm can be explored further by adding more features, learned by a deep learning model."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}