0700.Model-complexity.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model complexity\n",
    "\n",
    "In the last notebook, we saw that we could customize a model by adding a parameter. Doing so, we were able to fit that model to a data point. This fit was perfect, insofar as numerics would allow.\n",
    "\n",
    "In the next notebook, we'll see that as we add more data to our data set, fitting a model to our data usually becomes more challenging and the result will be less perfect.\n",
    "\n",
    "For one thing, we will find that we can add complexity to our model to capture added complexity in the data. We can do this by adding more parameters to our model. We'll see that for a data set with two data points, we can again get a \"perfect\" fit to our model by adding a second parameter to our model.\n",
    "\n",
    "However, we can't simply add a parameter to our model every time we add a data point to our data set, since this will lead to a phenomenon called **overfitting**.\n",
    "\n",
    "In the image below, we depict a data set that is close to linear, and models that exhibit underfitting, fitting well, and overfitting, from left to right:\n",
    "\n",
    "<img src=\"https://raw.githubusercontent.com/JuliaComputing/JuliaAcademyData.jl/master/courses/Foundations%20of%20machine%20learning/data/model_fitting.png\" alt=\"Drawing\" style=\"width: 800px;\"/>\n",
    "\n",
    "\n",
    "In the first image, the model accounts for the slope along which the data falls, but not the offset.\n",
    "\n",
    "In the second image, the model accounts for both the slope and offset of the  data. Adding this second parameter (the offset) to the model creates a much better fit.\n",
    "\n",
    "However, we can imagine that a model can have too many parameters, where we begin to fit  not only the high level features of the data, but also the noise. This overfitting is depicted in the third image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our aim will be to fit the data well, but avoiding *over*fitting the data!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.6.0",
   "language": "julia",
   "name": "julia-1.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}