{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = [1, 2, 3]\n",
    "type(x)\n",
    "# list\n",
    "\n",
    "y = [\"a\", \"b\", \"c\"]\n",
    "z = [True, False, False, True]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "::: {.callout-warning}\n",
    "**Watch out!** Python indexing starts at **0**, not 1 as in R!\n",
    ":::"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y[0]   # 'a'   <- first element\n",
    "y[1]   # 'b'   <- second element"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lists: Printing, Concatenating, Coercing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"The first value in y is '\", y[0], \"'.\", sep=\"\")\n",
    "# The first value in y is 'a'.\n",
    "\n",
    "\"The first value in y is \" + y[0] + \".\"\n",
    "# 'The first value is y is a.'\n",
    "\n",
    "str(x[0])   # '1'  -- coerce a number to a string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A list can build a sequence with `range`, and can mix types freely:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u = list(range(0, 10, 2))   # [0, 2, 4, 6, 8]\n",
    "v = [True, 0, \"whatever\", [1, 2]]\n",
    "type(v[2])   # str"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lists: Multiple Assignment and Editing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t, u, v = 3, [4, 5], \"hello\"\n",
    "\n",
    "tuv = [t, u, v]\n",
    "tuv[2] = \"good-bye\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Important contrast with R:** the `*` operator on a list does *not* do entrywise arithmetic --- it repeats the list!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x * 2\n",
    "# [1, 2, 3, 1, 2, 3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is one of several signs that lists are not built for numerical computing --- that's what NumPy arrays are for.\n",
    "\n",
    "## The Python Tuple\n",
    "\n",
    "A **tuple** is like a list, but **immutable** --- it cannot be edited after creation. Built with parentheses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "salutations = (\"hey\", \"hi\", \"ahoy\", \"sup\")\n",
    "salutations[0]     # 'hey'\n",
    "\n",
    "list(salutations)  # convert to a list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Trying to edit a tuple's entry raises an error\n",
    "- Immutability makes tuples more memory-efficient\n",
    "- We will mostly see tuples used as *arguments* to functions (e.g. specifying the shape of an array)\n",
    "\n",
    "## The Python Dictionary\n",
    "\n",
    "A **dictionary** stores **key--value** pairs, accessed by key rather than by position:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stat_540 = {'nstudents'  : 29,\n",
    "            'time'       : '2:20 - 3:35 pm',\n",
    "            'days'       : ['Tue', 'Thu'],\n",
    "            'instructor' : 'Dr. Huang'}\n",
    "\n",
    "stat_540['time']    # '1:15 - 2:30 pm'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Values can be edited by key:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stat_540['instructor'] = 'Dr. Ho'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For most of our statistics and data-science work, we will rely much more on **NumPy arrays** than on lists or dictionaries.\n",
    "\n",
    "## Outline\n",
    "\n",
    "1. Why Python? Getting oriented\n",
    "2. Basic Python objects: lists, tuples, dictionaries\n",
    "3. **NumPy arrays: creating, indexing, slicing**\n",
    "4. Array attributes, reshaping, and combining arrays\n",
    "5. Arithmetic and summary statistics on arrays\n",
    "6. Practice\n",
    "\n",
    "## Creating NumPy Arrays\n",
    "\n",
    "NumPy arrays are more like R's vectors and matrices: entries must all be the **same type**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "x = np.array([0, 1, 2])\n",
    "type(x)   # numpy.ndarray"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Building sequences, similarly to `seq()` in R:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-1.   -0.75 -0.5  -0.25  0.    0.25  0.5   0.75]\n",
      "[0.   0.05 0.1  0.15 0.2  0.25 0.3  0.35 0.4  0.45 0.5  0.55 0.6  0.65\n",
      " 0.7  0.75 0.8  0.85 0.9  0.95 1.  ]\n"
     ]
    }
   ],
   "source": [
    "seq  = np.arange(-1, 1, 1/4)   # start, stop, step\n",
    "seq2 = np.linspace(0, 1, 21)   # 21 equally-spaced points on [0,1]\n",
    "print(seq)\n",
    "print(seq2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mixed types get **upcast** (e.g. integers become floats) so the array stays a single type.\n",
    "\n",
    "## Creating Arrays from Scratch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.14820046, 0.62494951, 0.46134412],\n",
       "       [0.7677214 , 0.04497157, 0.53774888],\n",
       "       [0.41806187, 0.06130608, 0.1103631 ]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.zeros(10, dtype=int)     # length-10 array of zeros\n",
    "np.ones((3, 5))             # 3x5 array of ones\n",
    "np.full((3, 7), 4)          # 3x7 array, all entries = 4\n",
    "np.eye(3)                   # 3x3 identity matrix\n",
    "np.diag(np.ones(5))         # 5x5 diagonal / identity matrix\n",
    "\n",
    "np.random.random((3, 3))          # Uniform(0,1) entries\n",
    "np.random.normal(0, 1, (3, 3))    # Normal(0,1) entries\n",
    "np.random.randint(0, 10, (3, 3))  # random integers in [0,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Give `(rows, columns)` as a **tuple** for shape\n",
    "- A matrix (2-d array) is made from a list of equal-length lists: `np.array([[1,2,3],[4,5,6]])`\n",
    "\n",
    "## Random Number Generators\n",
    "\n",
    "The currently recommended way to draw random numbers is via a **generator** object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "rng = np.random.default_rng()\n",
    "\n",
    "X = rng.poisson(lam=2, size=20)\n",
    "X = rng.poisson(lam=2, size=(10, 3))  # tuple size -> 2-d array\n",
    "U = rng.random((4, 3))                # Uniform(0,1)\n",
    "B = rng.binomial(1, 1/2, (4, 3))      # Bernoulli(1/2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This mirrors `rpois()`, `runif()`, `rbinom()` in R, but the distribution's parameters are accessed as *methods* of the generator object.\n",
    "\n",
    "## Accessing Array Entries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-1.   -0.75 -0.5  -0.25  0.    0.25  0.5   0.75]\n",
      "-1.0\n",
      "[-0.5  -0.25]\n",
      "[-0.25  0.    0.25  0.5   0.75]\n",
      "[-1.   -0.75 -0.5  -0.25]\n",
      "[-1.   -0.25  0.5 ]\n",
      "0.75\n"
     ]
    }
   ],
   "source": [
    "seq = np.arange(-1, 1, 1/4)\n",
    "print(seq)\n",
    "print(seq[0])     # first entry: -1.0\n",
    "print(seq[2:4])   # entries with index 2, 3\n",
    "print(seq[3:])    # from index 3 to the end\n",
    "print(seq[:4])    # from the start up to (not incl.) index 4\n",
    "print(seq[::3])   # every third entry\n",
    "print(seq[-1])    # last entry (negative index counts from the end)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "::: {.callout-note}\n",
    "**Key difference from R:** In Python, a negative index counts entries from\n",
    "the *end*; it does *not* drop an entry as it does in R.\n",
    ":::\n",
    "\n",
    "## Indexing 2-D Arrays (Matrices)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2 3]\n",
      " [4 5 6]]\n",
      "[1 2 3]\n",
      "[2 5]\n",
      "6\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[1, 2, 3],\n",
       "       [4, 5, 7]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "M = np.array([[1, 2, 3], [4, 5, 6]])\n",
    "print(M)\n",
    "print(M[0, :])   # first row:    [1 2 3]\n",
    "print(M[:, 1])   # second column: [2 5]\n",
    "print(M[1, 2])   # single entry:  6\n",
    "M[1, 2] = 7 # replace an entry\n",
    "M"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De-selecting rows/columns uses `~` rather than a negative index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4 5 7]\n",
      "[[False  True False]\n",
      " [ True False False]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([1, 3, 5, 7])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(M[~0, :])   # drop the first row\n",
    "even = M % 2 == 0\n",
    "print(even)\n",
    "M[~even]          # boolean mask: keep the odd entries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outline\n",
    "\n",
    "1. Why Python? Getting oriented\n",
    "2. Basic Python objects: lists, tuples, dictionaries\n",
    "3. NumPy arrays: creating, indexing, slicing\n",
    "4. **Array attributes, reshaping, and combining arrays**\n",
    "5. Arithmetic and summary statistics on arrays\n",
    "6. Practice\n",
    "\n",
    "## Array Attributes\n",
    "\n",
    "Attributes describe an array's dimensions and storage type; access with a `.` (no parentheses):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('int64')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.random.randint(10, size=(3, 4, 5))\n",
    "\n",
    "A.ndim     # number of dimensions: 3\n",
    "A.shape    # size of each dimension: (3, 4, 5)\n",
    "A.size     # total number of entries: 60\n",
    "A.dtype    # data type, e.g. dtype('int64')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every NumPy array has a single **dtype**: common ones are `int64`, `float64`, `bool_`.\n",
    "\n",
    "::: {.callout-warning}\n",
    "**Watch out!** Assigning a float into an integer array silently truncates\n",
    "the value --- no warning is given!\n",
    ":::\n",
    "\n",
    "## Reshaping Arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.  , 0.05, 0.1 ],\n",
       "       [0.15, 0.2 , 0.25],\n",
       "       [0.3 , 0.35, 0.4 ],\n",
       "       [0.45, 0.5 , 0.55],\n",
       "       [0.6 , 0.65, 0.7 ],\n",
       "       [0.75, 0.8 , 0.85],\n",
       "       [0.9 , 0.95, 1.  ]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.reshape(np.linspace(0, 1, 21), (3, 7))\n",
    "np.reshape(np.linspace(0, 1, 21), (7, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `reshape` fills the new array **across rows** first\n",
    "- Use `np.transpose(A)` (or `A.transpose()`) to fill across columns instead\n",
    "- Arrays can have more than 2 dimensions, just as in R"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.transpose(np.reshape(np.linspace(0, 1, 21), (3, 7)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Methods: Applying Functions to Objects\n",
    "\n",
    "Many NumPy functions can also be called as a **method** attached to the object with a `.`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B = np.array([[1, 2], [3, 4]])\n",
    "\n",
    "np.transpose(B)   # function form\n",
    "B.transpose()     # method form -- same result\n",
    "\n",
    "np.sum(B)         # function form\n",
    "B.sum()           # method form -- same result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This \"`object.method()`\" pattern has no direct analogue in base R and shows up constantly in Python.\n",
    "\n",
    "## Aliases vs. Copies\n",
    "\n",
    "A **subset** of an array is an **alias**, not a new array --- editing it edits the original!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,   1,   2,   3,   4,   5],\n",
       "       [  6, -99,   8,   9,  10,  11],\n",
       "       [ 12,  13,  14,  15,  16,  17],\n",
       "       [ 18,  19,  20,  21,  22,  23]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.reshape(np.arange(24), (4, 6))\n",
    "A0 = A[:2, :2]\n",
    "A0[1, 1] = -99\n",
    "A\n",
    "# A has ALSO changed!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get an independent copy, append `.copy()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,   1,   2,   3,   4,   5],\n",
       "       [  6, -99,   8,   9,  10,  11],\n",
       "       [ 12,  13,  14,  15,  16,  17],\n",
       "       [ 18,  19,  20,  21,  22,  23]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A0_copy = A[:2, :2].copy()\n",
    "A0_copy[0, 0] = -99   # does NOT affect A\n",
    "A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Subsetting with a boolean mask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0,  0,  0,  0,  0,  5],\n",
       "       [ 6,  0,  8,  9, 10, 11],\n",
       "       [12, 13, 14, 15, 16, 17],\n",
       "       [18, 19, 20, 21, 22, 23]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A[A < 5] = 0 # set to zero all entries in A which are less than 0.5\n",
    "A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combining and Splitting Arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1,   2,   3,  98,  99, 100])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = np.array([1, 2, 3])\n",
    "y = np.array([98, 99, 100])\n",
    "np.concatenate([x, y])     # [1 2 3 98 99 100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 1 0]\n",
      " [0 0 1]\n",
      " [0 1 0]\n",
      " [0 1 0]]\n",
      "[[0.75381236 0.99570973 0.93179539]\n",
      " [0.66930078 0.25118955 0.9605348 ]]\n",
      "[[0 1 0]\n",
      " [0 1 0]]\n"
     ]
    }
   ],
   "source": [
    "u = rng.binomial(1,1/2,(4,3)) # 4 by 3\n",
    "v = rng.random((2,3))         # 2 by 3\n",
    "print(u)\n",
    "print(v)\n",
    "print(u[2:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.        , 1.        , 0.        , 0.75381236, 0.99570973,\n",
       "        0.93179539],\n",
       "       [0.        , 1.        , 0.        , 0.66930078, 0.25118955,\n",
       "        0.9605348 ]])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.vstack([u,v])               # 6 by 3\n",
    "np.vstack([u, v])          # stack rows (matching # cols)\n",
    "np.hstack([u[2:], v])          # stack columns (matching # rows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Splitting is the reverse operation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1 2]\n",
      "[3 4 5]\n",
      "[ 6  7  8  9 10 11]\n"
     ]
    }
   ],
   "source": [
    "x1, x2, x3 = np.split(np.arange(12), [3, 6])\n",
    "print(x1)\n",
    "print(x2)\n",
    "print(x3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[3 2 2]\n",
      " [4 4 0]\n",
      " [4 3 2]\n",
      " [0 1 6]\n",
      " [3 3 1]\n",
      " [4 1 1]\n",
      " [5 1 3]\n",
      " [0 0 2]\n",
      " [1 1 0]\n",
      " [1 0 1]]\n",
      "[[3 2]\n",
      " [4 4]\n",
      " [4 3]\n",
      " [0 1]\n",
      " [3 3]\n",
      " [4 1]\n",
      " [5 1]\n",
      " [0 0]\n",
      " [1 1]\n",
      " [1 0]]\n",
      "[[2]\n",
      " [0]\n",
      " [2]\n",
      " [6]\n",
      " [1]\n",
      " [1]\n",
      " [3]\n",
      " [2]\n",
      " [0]\n",
      " [1]]\n"
     ]
    }
   ],
   "source": [
    "print(X)\n",
    "X1, X2 = np.hsplit(X, [2])   # split a matrix by column\n",
    "print(X1)\n",
    "print(X2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[3 2 2]\n",
      " [4 4 0]]\n",
      "[[4 3 2]\n",
      " [0 1 6]\n",
      " [3 3 1]\n",
      " [4 1 1]\n",
      " [5 1 3]\n",
      " [0 0 2]\n",
      " [1 1 0]\n",
      " [1 0 1]]\n"
     ]
    }
   ],
   "source": [
    "X1, X2 = np.vsplit(X, [2])   # split a matrix by row\n",
    "print(X1)\n",
    "print(X2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outline\n",
    "\n",
    "1. Why Python? Getting oriented\n",
    "2. Basic Python objects: lists, tuples, dictionaries\n",
    "3. NumPy arrays: creating, indexing, slicing\n",
    "4. Array attributes, reshaping, and combining arrays\n",
    "5. **Arithmetic and summary statistics on arrays**\n",
    "6. Practice\n",
    "\n",
    "## Arithmetic on NumPy Arrays\n",
    "\n",
    "The operators `+ - * /` and `**` (exponent), `%` (modulus) work **entrywise**, as in R:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 2])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([3, 4, 5])\n",
    "c = 2\n",
    "a + c    # [5 6 7]\n",
    "a ** c   # [9 16 25]   ('^' is NOT exponentiation in Python!)\n",
    "a // c   # floor divide: [1 2 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "::: {.callout-warning}\n",
    "**No recycling!** Unlike R, NumPy will *not* silently recycle a shorter\n",
    "array to match a longer one --- mismatched shapes raise an error.\n",
    ":::\n",
    "\n",
    "Common math functions live in NumPy: `np.exp`, `np.log`, `np.sin`, `np.arctan`, ...\n",
    "\n",
    "## Summary Statistics on Arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.5418694  0.86345938 0.83575667]\n",
      " [0.60095489 0.29872609 0.86011858]\n",
      " [0.52050281 0.65312051 0.84447221]\n",
      " [0.27952486 0.66706163 0.03845973]\n",
      " [0.3335868  0.07267842 0.15767958]\n",
      " [0.18332102 0.27963122 0.86957785]\n",
      " [0.0993152  0.91877771 0.63024339]\n",
      " [0.35705217 0.89061821 0.09820929]\n",
      " [0.81321854 0.55054865 0.2869501 ]\n",
      " [0.84586073 0.24006737 0.47574641]\n",
      " [0.084958   0.84284302 0.23467055]\n",
      " [0.97358104 0.12023211 0.27495485]\n",
      " [0.93972907 0.20070933 0.02107771]\n",
      " [0.77444258 0.98951662 0.36468564]\n",
      " [0.73612506 0.61433366 0.5601603 ]\n",
      " [0.24783695 0.40715013 0.66128099]\n",
      " [0.50456802 0.4511689  0.02111772]\n",
      " [0.59596104 0.17450243 0.20721927]\n",
      " [0.75262739 0.90853733 0.25274988]\n",
      " [0.49200894 0.44497315 0.3215491 ]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.9895166159164452"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "D = rng.random((20, 3))\n",
    "print(D)\n",
    "np.sum(D)     # or D.sum()\n",
    "np.min(D)     # or D.min()\n",
    "np.max(D)     # or D.max()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 8.73349453,  8.4674452 , 12.18563252])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "D.sum(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.61810602, 0.81397754, 0.5088196 , 0.94443434, 0.78419807,\n",
       "       0.99668802, 0.79078276, 0.92715229, 0.8358856 , 0.69289495,\n",
       "       0.8427604 , 0.84561569, 0.96577913, 0.75940195, 0.56301702,\n",
       "       0.7367655 , 0.52794625, 0.55931858, 0.67722432, 0.88355547])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "D = rng.random((20, 3))\n",
    "\n",
    "np.sum(D)     # or D.sum()\n",
    "np.min(D)     # or D.min()\n",
    "np.max(D)     # or D.max()\n",
    "\n",
    "D.sum(axis=0)   # column sums\n",
    "D.max(axis=1)   # row maxima"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `axis=0` operates **down** columns; `axis=1` operates **across** rows\n",
    "- `np.sum` is much faster than Python's built-in `sum` on large arrays\n",
    "\n",
    "## Missing Values\n",
    "\n",
    "Create a missing value with `None`; most summary functions have a `nan`-aware version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20, 3)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "size=D.shape\n",
    "size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 1],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [1, 0, 1],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [0, 1, 0]])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rng.binomial(1, 0.1, size=D.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.5418694  0.86345938 0.83575667]\n",
      " [0.60095489 0.29872609 0.86011858]\n",
      " [0.52050281 0.65312051 0.84447221]\n",
      " [0.27952486 0.66706163        nan]\n",
      " [0.3335868  0.07267842 0.15767958]\n",
      " [       nan 0.27963122        nan]\n",
      " [       nan        nan 0.63024339]\n",
      " [0.35705217 0.89061821 0.09820929]\n",
      " [0.81321854        nan 0.2869501 ]\n",
      " [0.84586073 0.24006737 0.47574641]\n",
      " [0.084958   0.84284302 0.23467055]\n",
      " [0.97358104 0.12023211 0.27495485]\n",
      " [0.93972907 0.20070933 0.02107771]\n",
      " [       nan        nan 0.36468564]\n",
      " [       nan        nan        nan]\n",
      " [0.24783695 0.40715013 0.66128099]\n",
      " [       nan 0.4511689  0.02111772]\n",
      " [       nan 0.17450243 0.20721927]\n",
      " [0.75262739 0.90853733 0.25274988]\n",
      " [0.49200894 0.44497315 0.3215491 ]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([0.084958  , 0.07267842, 0.02107771])"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "D[rng.binomial(1, 0.1, size=D.shape) == 1] = None\n",
    "print(D)\n",
    "np.nanmax(D)          # ignores missing values\n",
    "np.nanmin(D, axis=0)  # column minima, ignoring NaNs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare: `np.sum` $\\to$ `np.nansum`, `np.mean` $\\to$ `np.nanmean`, etc. --- the same pattern as `na.rm = TRUE` in R, just with a different function name instead of an argument.\n",
    "\n",
    "## Outline\n",
    "\n",
    "1. Why Python? Getting oriented\n",
    "2. Basic Python objects: lists, tuples, dictionaries\n",
    "3. NumPy arrays: creating, indexing, slicing\n",
    "4. Array attributes, reshaping, and combining arrays\n",
    "5. Arithmetic and summary statistics on arrays\n",
    "6. **Practice**\n",
    "\n",
    "## Practice: Write Code\n",
    "\n",
    "1. Write code to create the list `[0, 3, 6, 9, 12, 0, 3, 6, 9, 12]`.\n",
    "\n",
    "2. Create the NumPy array whose $i$th row (starting at $i=0$) is filled with the value $i$, repeated 4 times, for $i = 0, \\dots, 7$.\n",
    "\n",
    "3. Write code to build the $(n-1) \\times n$ \"successive difference\" matrix with $-1$ on the main diagonal and $1$ on the diagonal just above it, and zeros elsewhere, for any $n$.\n",
    "\n",
    "4. Simulate 10,000 rolls of a pair of six-sided dice with `rng.integers(low=1, high=7, size=(10000,2))`. Find the proportion of rolls whose sum is odd.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 3, 6, 9, 12, 0, 3, 6, 9, 12]"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "x=list(np.arange(0,14, 3))*2\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0 0 0 0]\n",
      " [1 1 1 1]\n",
      " [2 2 2 2]\n",
      " [3 3 3 3]\n",
      " [4 4 4 4]\n",
      " [5 5 5 5]\n",
      " [6 6 6 6]\n",
      " [7 7 7 7]]\n"
     ]
    }
   ],
   "source": [
    "A=np.zeros((8,4), dtype=int)\n",
    "for i in range(8): \n",
    "    A[i, :] = i\n",
    "print(A)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = np.array([[i] * 4 for i in range(8)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0 0 0 0]\n",
      " [1 1 1 1]\n",
      " [2 2 2 2]\n",
      " [3 3 3 3]\n",
      " [4 4 4 4]\n",
      " [5 5 5 5]\n",
      " [6 6 6 6]\n",
      " [7 7 7 7]]\n"
     ]
    }
   ],
   "source": [
    "A = np.arange(8).repeat(4).reshape(8, 4)\n",
    "print(A)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0.]])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n=4\n",
    "D=np.zeros((n-1, n))\n",
    "D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1.,  1.,  0.,  0.],\n",
       "       [ 0., -1.,  1.,  0.],\n",
       "       [ 0.,  0., -1.,  1.]])"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.fill_diagonal(D, -1) \n",
    "D\n",
    "np.fill_diagonal(D[:, 1:], 1)\n",
    "D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-1.  1.  0.  0.  0.]\n",
      " [ 0. -1.  1.  0.  0.]\n",
      " [ 0.  0. -1.  1.  0.]\n",
      " [ 0.  0.  0. -1.  1.]]\n"
     ]
    }
   ],
   "source": [
    "def diff_matrix(n):\n",
    "    D = np.zeros((n - 1, n))\n",
    "    np.fill_diagonal(D, -1)           # -1 on main diagonal\n",
    "    np.fill_diagonal(D[:, 1:], 1)     # +1 on diagonal just above\n",
    "    return D\n",
    "\n",
    "print(diff_matrix(5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 2)"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rolls = rng.integers(low=1, high=7, size=(10000, 2))   # shape (10000, 2)\n",
    "rolls.shape\n",
    "                              # sum each pair\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Proportion of odd sums: 0.5041\n"
     ]
    }
   ],
   "source": [
    "totals = rolls.sum(axis=1)  \n",
    "totals\n",
    "prop_odd = np.mean(totals % 2 == 1)\n",
    "print(f\"Proportion of odd sums: {prop_odd:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice: Read Code\n",
    "\n",
    "Predict the output of each code chunk before running it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'cow'"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# (1)\n",
    "ch = \"Why hello.\"\n",
    "ch[2:]\n",
    "\n",
    "# (2)\n",
    "x = list(range(0, 15, 3))\n",
    "x * 2\n",
    "\n",
    "# (3)\n",
    "a, b, c = [4, 5], ['cat', 'cow'], [True, False]\n",
    "d = [a, b, c]\n",
    "d[1][1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (genomics-cnn)",
   "language": "python",
   "name": "genomics-cnn"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.25"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}