{ "cells": [ { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "
\n", "As a reminder, one of the prerequisites for this course if programming experience, especially in Python. If you do not have experience in Python specifically, we strongly recommend you go through the Codecademy Python course as soon as possible to brush up on the basics of Python.\n", "
" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "
Before going through this notebook, you may want to take a quick look at [7 - Debugging.ipynb](7 - Debugging.ipynb) if you haven't already for some tips on debugging your code when you get stuck.
" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "Sometimes, there are more advanced operations we want to do with NumPy arrays. For example, if we had an array of values and wanted to set all negative values to zero, how would we do this? The answer is called *fancy indexing*, and be done two ways: boolean indexing, and array indexing." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "import numpy as np" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "## Boolean indexing" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "The idea behind boolean indexing is that for each element of the array, we know whether we want to select it or not. A *boolean array* is an array of the same shape as our original array which contains only True and False values. The location of the True values in our boolean array indicate the location of the element in our original array that we want to select, while the location of the False values correspond to those elements in our original array that we _don't_ want to select.\n", "\n", "Let's consider our experiment data again:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "data = np.load(\"data/experiment_data.npy\")\n", "data" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "Recall that these are reaction times. It is typically accepted that really low reaction times -- such as less than 100 milliseconds -- are too fast for people to have actually seen and processed the stimulus. Let's see if there are any reaction times less than 100 milliseconds in our data.\n", "\n", "To pull out just the elements less than 100 milliseconds, we need two steps. First, we use boolean comparisons to check which are less than 100ms:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "too_fast = data < 100\n", "too_fast" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "Then, using this `too_fast` array, we can index *back into* the original array, and see that there are indeed some trials which were abnormally fast:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "data[too_fast]" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "What this is doing is essentially saying: for every element in `too_fast` that is `True`, give me the corresponding element in `arr`. \n", "\n", "Bcause this is a boolean array, we can also negate it, and pull out all the elements that we consider to be valid reaction times:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "data[~too_fast]" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "Not only does this *give* you the elements, but modifying those elements will modify the original array, too. In this case, we will set our \"too fast\" elements to have a value of \"not a number\", or `NaN`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "data[too_fast] = np.nan\n", "data" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "Now, if we try to find which elements are less than 100 milliseconds, we will not find any:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "data[data < 100]" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "
Note: You may see a RuntimeWarning when you run the above cell, saying that an \"invalid value\" was encountered. Sometimes, it is possible for NaNs to appear in an array without your knowledge: for example, if you multiply infinity (np.inf) by zero. So, NumPy is warning us that it has encountered NaNs (the \"invalid value\") in case we weren't aware. We knew there were NaNs because we put them there, so in this scenario we can safely ignore the warning. However, if you encounter a warning like this in the future and you weren't expecting it, make sure you investigate the source of the warning!\n", "
" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "### Exercise: Threshold (2 points)" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "
\n", "Write a function, threshold, which takes an array and returns a new array with values thresholded by the mean of the array.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "nbgrader": { "grade": false, "grade_id": "threshold", "locked": false, "schema_version": 1, "solution": true } }, "outputs": [], "source": [ "def threshold(arr):\n", " \"\"\"Computes the mean of the given array, and returns a new array which\n", " is 1 where values in the original array are greater than the mean, 0 where\n", " they are equal to the mean, and -1 where they are less than the mean.\n", "\n", " Remember that if you want to create a copy of an array, you need to use\n", " `arr.copy()`.\n", " \n", " Hint: your solution should use boolean indexing, and can be done in six\n", " lines of code (including the return statement).\n", " \n", " Parameters\n", " ----------\n", " arr : numpy.ndarray\n", "\n", " Returns\n", " -------\n", " new_arr : thresholded version of `arr`\n", " \n", " \"\"\"\n", " ### BEGIN SOLUTION\n", " array_mean = np.mean(arr)\n", " new_arr = arr.copy()\n", " new_arr[arr > array_mean] = 1\n", " new_arr[arr == array_mean] = 0\n", " new_arr[arr < array_mean] = -1\n", " return new_arr\n", " ### END SOLUTION" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "# add your own test cases in this cell!\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": true, "grade_id": "test_threshold_correct", "locked": false, "points": 1.5, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "\"\"\"Try a few obvious threshold cases.\"\"\"\n", "from numpy.testing import assert_array_equal\n", "assert_array_equal(threshold(np.array([1, 1, 1, 1])), np.array([0, 0, 0, 0]))\n", "assert_array_equal(threshold(np.array([1, 0, 1, 0])), np.array([1, -1, 1, -1]))\n", "assert_array_equal(threshold(np.array([1, 0.5, 0, 0.5])), np.array([1, 0, -1, 0]))\n", "assert_array_equal(\n", " threshold(np.array([[0.5, 0.2, -0.3, 0.1], [1.7, -3.8, 0.5, 0.6]])), \n", " np.array([[1, 1, -1, 1], [1, -1, 1, 1]]))\n", "print(\"Success!\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": true, "grade_id": "test_threshold_copy", "locked": false, "points": 0.5, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "\"\"\"Make sure a copy of the array is being returned, and that the original array is unmodified.\"\"\"\n", "x = np.array([[0.5, 0.2, -0.3, 0.1], [1.7, -3.8, 0.5, 0.6]])\n", "y = threshold(x)\n", "assert_array_equal(x, np.array([[0.5, 0.2, -0.3, 0.1], [1.7, -3.8, 0.5, 0.6]]))\n", "assert_array_equal(y, np.array([[1, 1, -1, 1], [1, -1, 1, 1]]))\n", "print(\"Success!\")" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "## Array indexing" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "The other type of fancy indexing is *array indexing*. Let's consider our average response across participants:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "data = np.load(\"data/experiment_data.npy\")\n", "avg_responses = np.mean(data, axis=1)\n", "avg_responses" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "And let's say we also know which element corresponds to which participant, through the following `participants` array:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "participants = np.load(\"data/experiment_participants.npy\")\n", "participants" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "In other words, the first element of `avg_responses` corresponds to the first element of `participants` (so participant 45), the second element of `avg_responses` was given by participant 39, and so on.\n", "\n", "Let's say we wanted to know what participants had the largest average response, and what participants had the smallest average response. To do this, we might try sorting the responses:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "np.sort(avg_responses)" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "However, we then don't know which responses correspond to which trials. A different way to do this would be to use `np.argsort`, which returns an array of indices corresponding to the sorted order of the elements, rather than the elements in sorted order:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "np.argsort(avg_responses)" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "What this says is that element 18 is the smallest response, element 42 is the next smallest response, and so on, all the way to element 24, which is the largest response:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "avg_responses[18]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "avg_responses[42]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "avg_responses[24]" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "To use fancy indexing, we can actually use this array of integers as an index. If we use it on the original array, then we will obtain the sorted elements:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "avg_responses[np.argsort(avg_responses)]" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "And if we use it on our array of participants, then we can determine what participants had the largest and smallest responses:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "participants[np.argsort(avg_responses)]" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "So, in this case, participant 10 had the smallest average response, while participant 47 had the largest average response." ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "## From boolean to integer indices" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "Sometimes, we want to use a combination of boolean and array indexing. For example, if we wanted to pull out just the responses for participant 2, a natural approach would be to use boolean indexing:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "participant_2_responses = data[participants == 'p_002']" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "Another way that we could do this would be to determine the *index* of participant 2, and then use that to index into `data`. To do this, we can use a function called `np.argwhere`, which returns the indices of elements that are true:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "np.argwhere(participants == 'p_002')" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "So in this case, we see that participant 2 corresponds to index 26." ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "### Exercise: Averaging responses (2 points)" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "
\n", "Write a function that takes as arguments a participant id, the data, and the list of participant names, and computes the average response for the given participant.\n", "
" ] }, { "cell_type": "markdown", "metadata": { "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "source": [ "
Occasionally we will ask you to raise an error if your function gets inputs that it's not expecting. As a reminder, to raise an error, you should use the raise keyword. For example, to raise a ValueError, you would do raise ValueError(message), where message is a string explaining specifically what the error was.
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "nbgrader": { "grade": false, "grade_id": "participant_mean", "locked": false, "schema_version": 1, "solution": true } }, "outputs": [], "source": [ "def participant_mean(participant, data, participants):\n", " \"\"\"Computes the mean response for the given participant. A ValueError\n", " should be raised if more than one participant has the given name.\n", " \n", " Hint: your solution should use `np.argwhere`, and can be done in\n", " four lines (including the return statement).\n", " \n", " Parameters\n", " ----------\n", " participant: string\n", " The name/id of the participant\n", " data: numpy.ndarray with shape (n, m)\n", " Rows correspond to participants, columns to trials\n", " participants: numpy.ndarray with shape(n,)\n", " A string array containing participant names/ids, corresponding to\n", " the rows of the `data` array.\n", " \n", " Returns\n", " -------\n", " float: the mean response of the participant over all trials\n", " \n", " \"\"\"\n", " ### BEGIN SOLUTION\n", " i = np.argwhere(participants == participant)\n", " if i.size > 1:\n", " raise ValueError(\"more than one participant with id: \" + participant)\n", " return np.mean(data[i])\n", " ### END SOLUTION" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "nbgrader": { "grade": false, "locked": false, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "# add your own test cases in this cell!\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": true, "grade_id": "test_participant_mean_1", "locked": false, "points": 1.0, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "\"\"\"Check for correct answers with the example experiment data.\"\"\"\n", "from numpy.testing import assert_allclose\n", "data = np.load(\"data/experiment_data.npy\")\n", "participants = np.load(\"data/experiment_participants.npy\")\n", "assert_allclose(participant_mean('p_002', data, participants), 1857.7013113499095)\n", "assert_allclose(participant_mean('p_047', data, participants), 1906.0651466520821)\n", "assert_allclose(participant_mean('p_013', data, participants), 1718.4379910225193)\n", "print(\"Success!\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": true, "grade_id": "test_participant_mean_2", "locked": false, "points": 0.5, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "\"\"\"Check for correct answers for some different data.\"\"\"\n", "data = np.arange(32).reshape((4, 8))\n", "participants = np.array(['a', 'b', 'c', 'd'])\n", "assert_allclose(participant_mean('a', data, participants), 3.5)\n", "assert_allclose(participant_mean('b', data, participants), 11.5)\n", "assert_allclose(participant_mean('c', data, participants), 19.5)\n", "assert_allclose(participant_mean('d', data, participants), 27.5)\n", "print(\"Success!\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "nbgrader": { "grade": true, "grade_id": "test_participant_mean_raises", "locked": false, "points": 0.5, "schema_version": 1, "solution": false } }, "outputs": [], "source": [ "\"\"\"Check that a ValueError is raised when the participant name is not unique.\"\"\"\n", "from nose.tools import assert_raises\n", "data = np.arange(32).reshape((4, 8))\n", "participants = np.array(['a', 'b', 'c', 'a'])\n", "assert_raises(ValueError, participant_mean, 'a', data, participants)\n", "print(\"Success!\")" ] } ], "metadata": { "celltoolbar": "Create Assignment", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.4.0" } }, "nbformat": 4, "nbformat_minor": 0 }