lessons/python/ESPIN-07-func.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://csdms.colorado.edu/wiki/ESPIn2020\"><img style=\"float: center; width: 75%\" src=\"../../media/ESPIn.png\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Programming with Python\n",
    "## Creating Functions\n",
    "### minutes: 30\n",
    "\n",
    "> ## Learning Objectives\n",
    ">\n",
    "> *   Define a function that takes parameters.\n",
    "> *   Return a value from a function.\n",
    "> *   Test and debug a function.\n",
    "> *   Set default values for function parameters.\n",
    "> *   Explain why we should divide programs into small, single-purpose functions.\n",
    "\n",
    "At this point,\n",
    "we've written code to draw some interesting features in our topographical data,\n",
    "loop over all our data files to quickly draw these plots for each of them,\n",
    "and have Python make decisions based on what it sees in our data.\n",
    "But, our code is getting pretty long and complicated;\n",
    "what if we had thousands of datasets,\n",
    "and didn't want to generate a figure for every single one?\n",
    "Commenting out the figure-drawing code is a nuisance.\n",
    "Also, what if we want to use that code again,\n",
    "on a different dataset or at a different point in our program?\n",
    "Cutting and pasting it is going to make our code get very long and very repetitive,\n",
    "very quickly.\n",
    "We'd like a way to package our code so that it is easier to reuse,\n",
    "and Python provides for this by letting us define things called 'functions' -\n",
    "a shorthand way of re-executing longer pieces of code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import required packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy \n",
    "import matplotlib.pyplot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by defining a function `fahr_to_kelvin` that converts temperatures from Fahrenheit to Kelvin:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fahr_to_kelvin(temp):\n",
    "    return ((temp - 32) * (5/9)) + 273.15"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function definition opens with the word `def`,\n",
    "which is followed by the name of the function\n",
    "and a parenthesized list of parameter names.\n",
    "The [body](reference.html#function-body) of the function --- the\n",
    "statements that are executed when it runs --- is indented below the definition line.\n",
    "\n",
    "When we call the function,\n",
    "the values we pass to it are assigned to those variables\n",
    "so that we can use them inside the function.\n",
    "Inside the function,\n",
    "we use a [return statement](reference.html#return-statement) to send a result back to whoever asked for it.\n",
    "\n",
    "Let's try running our function.\n",
    "Calling our own function is no different from calling any other function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('freezing point of water:', fahr_to_kelvin(32))\n",
    "print('boiling point of water:', fahr_to_kelvin(212))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've successfully called the function that we defined,\n",
    "and we have access to the value that we returned.\n",
    "\n",
    "## Composing Functions\n",
    "\n",
    "Now that we've seen how to turn Fahrenheit into Kelvin,\n",
    "it's easy to turn Kelvin into Celsius:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def kelvin_to_celsius(temp_k):\n",
    "    return temp_k - 273.15\n",
    "\n",
    "print('absolute zero in Celsius:', kelvin_to_celsius(0.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What about converting Fahrenheit to Celsius?\n",
    "We could write out the formula,\n",
    "but we don't need to.\n",
    "Instead,\n",
    "we can [compose](reference.html#compose) the two functions we have already created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fahr_to_celsius(...):\n",
    "    ...\n",
    "\n",
    "print('freezing point of water in Celsius:', fahr_to_celsius(32.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is our first taste of how larger programs are built:\n",
    "we define basic operations,\n",
    "then combine them in ever-larger chunks to get the effect we want.\n",
    "Real-life functions will usually be larger than the ones shown here --- typically half a dozen to a few dozen lines --- but\n",
    "they shouldn't ever be much longer than that,\n",
    "or the next person who reads it won't be able to understand what's going on.\n",
    "\n",
    "## Tidying up\n",
    "\n",
    "Now that we know how to wrap bits of code up in functions,\n",
    "we can make our topo analysis easier to read and easier to reuse.\n",
    "First, let's make an `analyze` function that generates our plots:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def analyze(filename):\n",
    "\n",
    "    data = numpy.loadtxt(fname=filename, delimiter=',')\n",
    "\n",
    "    fig = matplotlib.pyplot.figure(figsize=(10.0, 3.0))\n",
    "\n",
    "    axes1 = fig.add_subplot(1, 3, 1)\n",
    "    axes2 = fig.add_subplot(1, 3, 2)\n",
    "    axes3 = fig.add_subplot(1, 3, 3)\n",
    "\n",
    "    axes1.set_ylabel('average')\n",
    "    axes1.plot(data.mean(axis=0))\n",
    "\n",
    "    axes2.set_ylabel('max')\n",
    "    axes2.plot(data.max(axis=0))\n",
    "\n",
    "    axes3.set_ylabel('min')\n",
    "    axes3.plot(data.min(axis=0))\n",
    "\n",
    "    fig.tight_layout()\n",
    "    matplotlib.pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try it: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename ='../../data/topo.asc'\n",
    "analyze(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and another function called `detect_problems` that checks for those systematics\n",
    "we noticed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def detect_problems(filename):\n",
    "\n",
    "    data = numpy.loadtxt(fname=filename, delimiter=',')\n",
    "\n",
    "    if data.max(axis=0)[0] == 0 and data.max(axis=0)[20] == 20:\n",
    "        print('Suspicious looking maxima!')\n",
    "    elif data.min(axis=0).sum() == 0:\n",
    "        print('Minima add up to zero!')\n",
    "    else:\n",
    "        print('Seems OK!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try it: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename ='../../data/topo.asc'\n",
    "detect_problems(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing and Documenting\n",
    "\n",
    "Once we start putting things in functions so that we can re-use them,\n",
    "we need to start testing that those functions are working correctly.\n",
    "To see how to do this,\n",
    "let's write a function to center a dataset around a particular value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def center(data, desired):\n",
    "    return (data - data.mean()) + desired"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could test this on our actual data,\n",
    "but since we don't know what the values ought to be,\n",
    "it will be hard to tell if the result was correct.\n",
    "Instead,\n",
    "let's use NumPy to create a matrix of 0's\n",
    "and then center that around 3:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = numpy.zeros((2,2))\n",
    "print(z)\n",
    "print(center(z, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That looks right,\n",
    "so let's try `center` on our real data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename ='../../data/topo.asc'\n",
    "\n",
    "data = numpy.loadtxt(fname=filename, delimiter=',')\n",
    "print(center(data, 0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's hard to tell from the default output whether the result is correct,\n",
    "but there are a few simple tests that will reassure us:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('original min, mean, and max are:', data.min(), data.mean(), data.max())\n",
    "centered = center(data, 0)\n",
    "print('min, mean, and and max of centered data are:', centered.min(), centered.mean(), centered.max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That seems almost right:\n",
    "the original mean was about ca. 3153.6,\n",
    "so the lower bound from zero is now about 3153.6-2565.0293.\n",
    "The mean of the centered data isn't quite zero --- we'll explore why not in the challenges --- but it's pretty close.\n",
    "We can even go further and check that the standard deviation hasn't changed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('std dev before and after:', data.std(), centered.std())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Those values look the same,\n",
    "but we probably wouldn't notice if they were different in the sixth decimal place.\n",
    "Let's do this instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('difference in standard deviations before and after:', data.std() - centered.std())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again,\n",
    "the difference is very small.\n",
    "It's still possible that our function is wrong,\n",
    "but it seems unlikely enough that we should probably get back to doing our analysis.\n",
    "We have one more task first, though:\n",
    "we should write some [documentation](reference.html#documentation) for our function\n",
    "to remind ourselves later what it's for and how to use it.\n",
    "\n",
    "The usual way to put documentation in software is to add [comments](reference.html#comment) like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# center(data, desired): return a new array containing the original data centered around the desired value.\n",
    "def center(data, desired):\n",
    "    return (data - data.mean()) + desired"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's a better way, though.\n",
    "If the first thing in a function is a string that isn't assigned to a variable,\n",
    "that string is attached to the function as its documentation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def center(data, desired):\n",
    "    '''Return a new array containing the original data centered around the desired value.'''\n",
    "    return (data - data.mean()) + desired"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is better because we can now ask Python's built-in help system to show us the documentation for the function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "help(center)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "attributes": {
     "classes": [
      "output"
     ],
     "id": ""
    }
   },
   "outputs": [],
   "source": [
    "?center"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A string like this is called a [docstring](reference.html#docstring).\n",
    "We don't need to use triple quotes when we write one,\n",
    "but if we do,\n",
    "we can break the string across multiple lines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def center(data, desired):\n",
    "    '''Return a new array containing the original data centered around the desired value.\n",
    "    Example: center([1, 2, 3], 0) => [-1, 0, 1]'''\n",
    "    return (data - data.mean()) + desired\n",
    "\n",
    "help(center)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining Defaults\n",
    "\n",
    "We have passed parameters to functions in two ways:\n",
    "directly, as in `type(data)`,\n",
    "and by name, as in `numpy.loadtxt(fname='something.csv', delimiter=',')`.\n",
    "In fact,\n",
    "we can pass the filename to `loadtxt` without the `fname=`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "numpy.loadtxt('../../data/topo.asc', delimiter=',')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "but we still need to say `delimiter=`:\n",
    "\n",
    "What happens if you enter the following statement?  \n",
    " ~~~ {.python}\n",
    " numpy.loadtxt('data/topo.asc', ',')\n",
    " ~~~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To understand what's going on,\n",
    "and make our own functions easier to use,\n",
    "let's re-define our `center` function like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def center(data, desired=0.0):\n",
    "    '''Return a new array containing the original data centered around the desired value (0 by default).\n",
    "    Example: center([1, 2, 3], 0) => [-1, 0, 1]'''\n",
    "    return (data - data.mean()) + desired"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The key change is that the second parameter is now written `desired=0.0` instead of just `desired`.\n",
    "If we call the function with two arguments,\n",
    "it works as it did before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data = numpy.zeros((2, 2))\n",
    "print(center(test_data, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But we can also now call it with just one parameter,\n",
    "in which case `desired` is automatically assigned the [default value](reference.html#default-value) of 0.0:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "more_data = 5 + numpy.zeros((2, 2))\n",
    "print('data before centering:')\n",
    "print(more_data)\n",
    "print('centered data:')\n",
    "print(center(more_data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is handy:\n",
    "if we usually want a function to work one way,\n",
    "but occasionally need it to do something else,\n",
    "we can allow people to pass a parameter when they need to\n",
    "but provide a default to make the normal case easier.\n",
    "The example below shows how Python matches values to parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def display(a=1, b=2, c=3):\n",
    "    print('a:', a, 'b:', b, 'c:', c)\n",
    "\n",
    "print('no parameters:')\n",
    "display()\n",
    "print('one parameter:')\n",
    "display(55)\n",
    "print('two parameters:')\n",
    "display(55, 66)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As this example shows,\n",
    "parameters are matched up from left to right,\n",
    "and any that haven't been given a value explicitly get their default value.\n",
    "We can override this behavior by naming the value as we pass it in:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('only setting the value of c')\n",
    "display(c=77)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With that in hand,\n",
    "let's look at the help for `numpy.loadtxt`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "help(numpy.loadtxt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's a lot of information here,\n",
    "but the most important part is the first couple of lines:\n",
    "\n",
    "~~~ {.output}\n",
    "loadtxt(fname, dtype=<type 'float'>, comments='#', delimiter=None, converters=None, skiprows=0, usecols=None,\n",
    "        unpack=False, ndmin=0)\n",
    "~~~\n",
    "\n",
    "This tells us that `loadtxt` has one parameter called `fname` that doesn't have a default value,\n",
    "and eight others that do.\n",
    "If we call the function like this (try):\n",
    " ~~~ {.python}\n",
    " numpy.loadtxt('data/topo.asc', ',')\n",
    " ~~~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "then the filename is assigned to `fname` (which is what we want),\n",
    "but the delimiter string `','` is assigned to `dtype` rather than `delimiter`,\n",
    "because `dtype` is the second parameter in the list. However ',' isn't a known `dtype` so\n",
    "our code produced an error message when we tried to run it.\n",
    "When we call `loadtxt` we don't have to provide `fname=` for the filename because it's the\n",
    "first item in the list, but if we want the ',' to be assigned to the variable `delimiter`,\n",
    "we *do* have to provide `delimiter=` for the second parameter since `delimiter` is not\n",
    "the second parameter in the list."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ## Test your skills 01: Combining strings \n",
    "\n",
    " \"Adding\" two strings produces their concatenation:\n",
    " `'a' + 'b'` is `'ab'`.\n",
    " Write a function called `fence` that takes two parameters called `original` and `wrapper`\n",
    " and returns a new string that has the wrapper character at the beginning and end of the original.\n",
    " A call to your function should look like this:\n",
    "\n",
    " ~~~ {.python}\n",
    " print(fence('name', '*'))\n",
    " ~~~\n",
    " ~~~ {.output}\n",
    " *name*\n",
    " ~~~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test your skills 02: Selecting characters from strings \n",
    "\n",
    " If the variable `s` refers to a string,\n",
    " then `s[0]` is the string's first character\n",
    " and `s[-1]` is its last.\n",
    " Write a function called `outer`\n",
    " that returns a string made up of just the first and last characters of its input.\n",
    " A call to your function should look like this:\n",
    "\n",
    " ~~~ {.python}\n",
    " print(outer('helium'))\n",
    " ~~~\n",
    " ~~~ {.output}\n",
    " hm\n",
    " ~~~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test your skills 03 : Rescaling an array \n",
    "\n",
    " Write a function `rescale` that takes an array as input\n",
    " and returns a corresponding array of values scaled to lie in the range 0.0 to 1.0.\n",
    " (Hint: If $L$ and $H$ are the lowest and highest values in the original array,\n",
    " then the replacement for a value $v$ should be $(v-L) / (H-L)$.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test your skills 04: Testing and documenting your function \n",
    "\n",
    " Run the commands `help(numpy.arange)` and `help(numpy.linspace)`\n",
    " to see how to use these functions to generate regularly-spaced values,\n",
    " then use those values to test your `rescale` function.\n",
    " Once you've successfully tested your function,\n",
    " add a docstring that explains what it does.\n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test your skills 05: Defining defaults \n",
    "\n",
    " Rewrite the `rescale` function so that it scales data to lie between 0.0 and 1.0 by default,\n",
    " but will allow the caller to specify lower and upper bounds if they want.\n",
    " Compare your implementation to your neighbor's:\n",
    " do the two functions always behave the same way?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test your skills 06: Variables inside and outside functions\n",
    "\n",
    " What does the following piece of code display when run - and why?\n",
    "\n",
    " ~~~ {.python}\n",
    " f = 0\n",
    " k = 0\n",
    "\n",
    " def f2k(f):\n",
    "   k = ((f-32)*(5.0/9.0)) + 273.15\n",
    "   return k\n",
    "\n",
    " f2k(8)\n",
    " f2k(41)\n",
    " f2k(32)\n",
    "\n",
    " print(k)\n",
    " ~~~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}