Day3.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "78695398",
   "metadata": {},
   "source": [
    "# Let's explore data visualization\n",
    "\n",
    "I will use a package called *pandas* and a python visualization package called *seaborn*. Seaborn (and *plotly*, which I may also use in the future) are both based on *matplotlib*.\n",
    "\n",
    "In your project, you may **not** use pandas. But you may use matplotlib."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "317303ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9b20ba21",
   "metadata": {},
   "source": [
    "Let's load and *visualize* the car data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "646d11e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = np.genfromtxt('data/vehiclesNumeric.csv', delimiter=',', skip_header=1, dtype=int, encoding='utf-8')\n",
    "columns = ['id', 'price', 'year', 'odometer']\n",
    "carsdf = pd.DataFrame(data, columns=columns)\n",
    "\n",
    "seaborn.pairplot(data=carsdf, x_vars=[\"price\"], y_vars=[\"year\"], height=5).set(title=\"Craigslist used car prices against car year of manufacture\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f5d42c41",
   "metadata": {},
   "source": [
    "* What is a pairplot?\n",
    "* What are some other types of visualization? (Hint: What happens if you go to https://seaborn.pydata.org/examples/index.html or type help(seaborn)?)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ecf56e6b",
   "metadata": {},
   "source": [
    "It's quite easy to make misleading visualizations! See: https://uxdesign.cc/a-beginners-guide-to-identifying-misleading-data-visualizations-d82a93211ac6"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2439171c",
   "metadata": {},
   "source": [
    "Be a good visualization creator!\n",
    "* use the right type of visualization (hint: pie charts are very rarely the right type of visualization!)\n",
    "* number and label your axes\n",
    "* provide an informative caption / label\n",
    "* unless there's a really good reason, start all axes at 0\n",
    "* use color schemes that work for people who are color blind\n",
    "\n",
    "\n",
    "\n",
    "Other resources:\n",
    "* https://uxdesign.cc/how-to-design-data-visualizations-that-are-actually-valuable-e8b752835b9a\n",
    "* https://www.tableau.com/about/blog/examining-data-viz-rules-dont-use-red-green-together\n",
    "* https://seaborn.pydata.org/tutorial/color_palettes.html"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fa8a394f",
   "metadata": {},
   "source": [
    "# Introduction to numpy: Making and looking at numpy arrays\n",
    "\n",
    "https://numpy.org/\n",
    "\n",
    "\n",
    "https://numpy.org/doc/stable/ <-- look here!!\n",
    "\n",
    "- the computer representation of linear alg\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40efba04",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "76fde5d8",
   "metadata": {},
   "source": [
    "## Data Types in numpy\n",
    "\n",
    "| Programming name | Math name |\n",
    "|------|------|\n",
    "| ?d array | vector |\n",
    "| ?d array | matrix |\n",
    "| ?d array | tensor |"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "87fee35e",
   "metadata": {},
   "source": [
    "GPUs and TPUs are set up to operate on vectors and matrices  \n",
    "\n",
    "*What is a GPU and what is a TPU?*\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "373da004",
   "metadata": {},
   "source": [
    "## Let's make vectors, matrices and tensors using numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b3b3ff3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's amke a vector\n",
    "v = np.array([1, 2, 3])\n",
    "print (v)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "107b7eb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's make a matrix\n",
    "m = np.array([[1,2,3], [4,5,6]])\n",
    "print(m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0513c19b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's make a tensor\n",
    "t = np.array([[[1,2,3,4], [5,6,7,8]], [[9,10,11,12], [13,14,15,16]],  [[17,18,19,20], [21,22,23,24]]])\n",
    "print(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a25f80f9",
   "metadata": {},
   "source": [
    "## Getting information about numpy arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "908414c4",
   "metadata": {},
   "source": [
    "### The type of a numpy array\n",
    "\n",
    "All the elements in a numpy array must have the same type (so you can't mix integers, floats, strings...)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59c7ae30",
   "metadata": {},
   "outputs": [],
   "source": [
    "# how do I figure out the type of a numpy array? \n",
    "print(\"v type\", v.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba6fd56c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# how do I change the type of a numpy array?\n",
    "vFloat = v.astype(float)\n",
    "print(\"vFloat type\", vFloat.dtype)\n",
    "print(\"vFloat\\n\", vFloat)\n",
    "\n",
    "vString = np.array(vFloat, dtype=str)\n",
    "print(\"vString type\", vString.dtype)\n",
    "#make new array, then tell them to be strings\n",
    "#coerce into strings for new numpy array\n",
    "print(\"vString\\n\", vString)\n",
    "\n",
    "vFloat = vString.astype(float)\n",
    "print(\"vFloat type\", vFloat.dtype)\n",
    "#try to coerce strings into floats\n",
    "print(\"vFloat\\n\", vFloat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed1b09e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# now you make it into an int array and print\n",
    "# going from float to int will truncate, so be careful with this"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a30f56da",
   "metadata": {},
   "source": [
    "### Getting information about the size of a numpy array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28a073b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's take a look at ndim, size, shape of v\n",
    "print(v.ndim)\n",
    "print(v.size) \n",
    "print(v.shape)\n",
    "print(v.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9575c625",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's take a look at ndim, size, shape of m\n",
    "print(m.ndim)\n",
    "print(m.size)\n",
    "print(m.shape)\n",
    "print(m.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4871ce62",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's take a look at ndim, size, shape of t\n",
    "print(t.ndim)\n",
    "print(t.size)\n",
    "print(t.shape) \n",
    "print(t.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd924421",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's save and load a numpy array\n",
    "import pickle as pkl\n",
    "\n",
    "# save to a pickle file\n",
    "with open('data.pkl','wb') as f:\n",
    "    pkl.dump(nparray, f)\n",
    "\n",
    "# load from a pickle file\n",
    "with open('data.pkl','rb') as f:\n",
    "    nparray2 = pkl.load(f)\n",
    "    print(nparray2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d84ff664",
   "metadata": {},
   "source": [
    "## From python lists to numpy arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e31450a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's make a numpy array from a python list\n",
    "pylist = [[10, 1, 2021], [2, 7, 2023]] \n",
    "nparray = np.array(pylist)\n",
    "print(\"python list\\n\", pylist)\n",
    "print(\"numpy array\\n\", nparray)\n",
    "print(\"shape\\n\", nparray.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a990c78a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's convert a numpy array to a python list\n",
    "backtolist = nparray.tolist()\n",
    "print(\"back to a python list\\n\", backtolist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62b5de1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# here's something we CANNOT do\n",
    "pylist = [[10, 1, 2021], [2, 7]] \n",
    "nparray = np.array(pylist)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dd1c8bc",
   "metadata": {},
   "source": [
    "*What is \"inhomogeneous\"?*\n",
    "\n",
    "*What must be true of a list we want to make into a numpy array?*"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a5b275e3",
   "metadata": {},
   "source": [
    "## Making numpy arrays without listing all the elements"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd51fe7f",
   "metadata": {},
   "source": [
    "### Zeros"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f29cd0aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# make an array of zeros\n",
    "nparrayZero = np.zeros([3, 10]) #3 by 10 nparray of zeros\n",
    "print(nparrayZero)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dab0ce46",
   "metadata": {},
   "outputs": [],
   "source": [
    "# that's floats ... what if we want ints?\n",
    "nparrayZero.astype(int) \n",
    "print(nparrayZero) #didn't work"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3e9e822",
   "metadata": {},
   "outputs": [],
   "source": [
    "nparrayZero = nparrayZero.astype(int) \n",
    "print(nparrayZero) #works"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c4cb01b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# another way to get ints\n",
    "nparrayZeroInt = np.array(nparrayZero, dtype=int)\n",
    "print(nparrayZeroInt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dfbe7c7",
   "metadata": {},
   "source": [
    "### Ones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5c45141",
   "metadata": {},
   "outputs": [],
   "source": [
    "# what if we want ones instead of zeros?\n",
    "\n",
    "nparrayOnes = np.ones([3,10])\n",
    "print(nparrayOnes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf52cdae",
   "metadata": {},
   "source": [
    "### More than ones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "252dfca3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# what if we want sevens?\n",
    "nparraySevens = ??\n",
    "print(nparraySevens)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "958d0a93",
   "metadata": {},
   "source": [
    "### Random values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dbdecd10",
   "metadata": {},
   "outputs": [],
   "source": [
    "# make an array of random values\n",
    "nparrayRandomFloat = np.random.random([3, 10])\n",
    "print(nparrayRandomFloat)\n",
    "print(nparrayRandomFloat.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2dfe0f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# what if we want random ints? let's see...\n",
    "nparrayRandomInt = np.random.random([3, 10], dtype=int) # can't do this because it generates random numbers between 0 and 1\n",
    "print(nparrayRandomInt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb1f2a3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# hmm, if not that then what?\n",
    "print(((nparrayRandomFloat)*10).astype(int))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "240da196",
   "metadata": {},
   "outputs": [],
   "source": [
    "# another way; let's take a look at the arguments here\n",
    "nparrayRandomInt = np.random.randint(1, 10, [3,7])\n",
    "print(nparrayRandomInt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e005daa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# yet another way!\n",
    "nplinArray = np.linspace(0,10,10, dtype=int).reshape(2,5) #ints but not random\n",
    "print(nplinArray)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9fdb5a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# now you try: what if we want 9 floats (not random) between .2 and .6, as a 3 x 3 numpy array?\n",
    "\n",
    "# what if we want to shape that into a 2 by 5 array?\n",
    "nplinArray = np.linspace(??).reshape(??)\n",
    "print(nplinArray)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  },
  "vscode": {
   "interpreter": {
    "hash": "3ad933181bd8a04b432d3370b9dc3b0662ad032c4dfaa4e4f1596c548f763858"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}