01_Overview_Forest_Fire_Prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e9248418-5829-4124-889e-a6414be78aa5",
   "metadata": {},
   "source": [
    "\n",
    "<figure>\n",
    "<img src=\"assets/ForestFireCartoon.jpg\" width=\"800\">\n",
    "<figcaption align = \"center\"> Image by pvproductions on Freepik: ”https://www.freepik.com/free-photo/fire-forest-flames-generative-ai_39872388.htm#query=forest%20fire&position=1&from_view=search&track=ais\"</figcaption>\n",
    "</figure>\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c2a05632-98e4-4de3-8b75-eb070ec93ed7",
   "metadata": {},
   "source": [
    "# Learning Objectives\n",
    "\n",
    "- Apply fine-tuning methodology to create a custom model for predicting forest fire likelihoods two years in advance\n",
    "- Describe Intel(r) Extension for PyTorch* (IPEX) and how it can be applied to access Intel* GPUs\n",
    "- Describe how IPEX can be used to accelerate a single model such as ResNet or a pipeline of models such as Stable Diffusion*\n",
    "- Apply code changes required to utilize Intel® Data Center GPU Max 1100 and how to accelerate your model using IPEX\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0bb1a2d2-c1a5-470b-bf8a-dd60ad5e132c",
   "metadata": {},
   "source": [
    "# Background\n",
    "\n",
    "This workshop, follows the article **[\"Predict Forest Fires using PyTorch\"](https://medium.com/@zmadscientist/predict-forest-fires-using-pytorch-333e3d6f01ac)**, by Bob Chesebrough.\n",
    "\n",
    "Forest fires very often cause damages and loss of human lives, ecosystems, and wildlife. Early identification of high fire likelihood areas is an important aid in early deployment of counter fire measures for prescribed burns or developing road access or other early preventative measures. \n",
    "\n",
    "AccuWeather Founder and CEO Dr. Joel N. Myers stated that the \"total damage and cumulative economic loss for the **2021 wildfire season** was expected to be between **\\\\$70 billion** and **\\\\$90 billion** in the U.S. with **\\\\$45 billion** to **\\\\$55 billion** of those damages to California alone\".  \n",
    "\n",
    "### A recent approach\n",
    "In a recent article, [interestingengineering.com: /a-newly-developed-ai-based-method-can-accurately-predict-wildfires](https://interestingengineering.com/science/a-newly-developed-ai-based-method-can-accurately-predict-wildfires ) , it was revealed that scientists from South Korea and the United States created a hybrid AI + weather forecast to predict extreme fire danger out to one week accuracy. Whereas, this model uses only aerial photos but shows promise up to 2 or three years in advance, at least for north central California. Researcher Simon Wang, from Utah State University, said of his model, **\"The AI-based predictions associated with extreme levels of fire danger are well grounded to strong winds and specific geographical characteristics, including high mountains and canyons in the Western United States that have been traditionally difficult to resolve with coarser models.\"**\n",
    "\n",
    "In this article, I propose using transfer learning using **PyTorch** to classify aerial photos according to the fire danger they convey using image details only. I use the **MODIS Burn Area dataset** to establish known fire burn areas in California from 2018 to 2020. I then sample aerial images acquired from **USDA/NAIP/DOQQ dataset** from the **prior two year period**, 2016 to 2017 in areas within and near the establish future fire regions. \n",
    "\n",
    "Then I use fine-tuning to adapt a pretrained **Resnet 18 model** (never trained on aerial photos) and use transfer learning on on either a **4th Gen Intel® Xeon processor** or on a **Intel® Data Center GPU Max Series**. The model is trained on a couple hundred images and labels for Fire and NoFire images. I then demonstrate how well the model works.  \n",
    "\n",
    "My model is more of a spatial-temporal model of sorts but more emphasis is on spatial prediction of where fires are likely to occur and then tested in the following 2 year interval.\n",
    "\n",
    "Traditional methods have been employed for many years to forecast these fires, but with new AI based algorithms, now we have the potential to enhance the predictability of forest fires. In this workshop, I will briefly explore the importance of predicting forest fires and then delve into my case study of using ResNet classifier in conjunction with the **MODIS Burn Area data set** to label future fire locations from the 2016 perspective.\n",
    "\n",
    "## Steps:\n",
    "\n",
    "1)\tNotebook number 1 - I describe how I used the **MODIS Burn Area dataset** to establish known fire burn areas in California from 2018 to 2020 and acquired **USDA/NAIP/DOQQ dataset** images to train on. I also describe why for the lab we are NOT using real data but synthetic data instead.\n",
    "2)\tNotebook number 2 - I take synthetic data we created prior to the workshop and \"Salt\" these images to be slightly more or less brown, and establish a faux Modis Burn area and faux image coordinates\n",
    "3)\tNotebook number 3 - I use fine-tuning to adapt a pre-trained **Resnet 18 model** (never trained on aerial photos) and use transfer learning on a **Intel® Data Center GPU Max Series GPU**. The model is trained on a couple hundred images and labels for Fire and NoFire images.\n",
    "4)\tNotebook number 4 - I score the newly trained model and generate a confusion matrix as well as a map showing where the mis-predicted images would align with respect to the faux burn area.\n",
    "5)\tNotebook number 5 -  I demonstrate how to generate the synthetic data using Stable Diffusion. I also demonstrate how to properly optimize the model/pipeline of models with the Intel(r) Extensions for PyTorch* to acheive good performance.\n",
    "6)\tNotebook number 6 -  I describe where and how you can download the actual images to predict real fires!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d32be5b-57b3-4256-ad3a-3909106c6aa0",
   "metadata": {},
   "source": [
    "# Sampling Method\n",
    "\n",
    "I used the Modis Burn datasets to identify which regions in CA had been burned in forest fires in the period from 2018 to 2020.\n",
    "\n",
    "I then collected images from these same regions from the period 2016 to end of 2017 and used the pre-burn images from both known fire and non-fire regions to make a ResNet 18 binary model to predict Fire or NoFire for a given image.\n",
    "\n",
    "The sampling index map is displayed below for both the Fire and NoFire cases showing the locations we sampled with aerial photos."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a64819a2-16a0-4c95-a4e8-974a24c41238",
   "metadata": {},
   "source": [
    "# Overview of Fine-tuning Project\n",
    "\n",
    "The purpose of this series of notebooks of Fi-netuning is to experiment with using aerial photos as a basis for predicting forest fire likilyhood based on the ModiBurn s datasets using **[Google Earth Engine](https://code.earthengine.google.com/)** or the **[USGS Earth Explorer](https://earthexplorer.usgs.gov/)** **USDA/NAIP/DOQQ aerial photos** and **MODIS Burn Area datasets** - all from within the State of California.\n",
    "\n",
    "For this workshop, however, in order to provide the data for the lab, we have provided synthetic data.\n",
    "\n",
    "By syntheic data, I mean I created faux images using Stable Diffusion to do images to image and text to images never before seen images! I will describe this process towards the end of the workshop.\n",
    "\n",
    "The discussion that follows in this notebook, is from actual data from NAIP from 2016-2017 and the Burn MODIS data is filtered to show the fires in CA from 2018 thru 2020.\n",
    "\n",
    "<figure>\n",
    "<img src=\"assets/ModisSampling.png\" width=\"600\">\n",
    "<figcaption align = \"center\"> Figure 1. Sampled locations used: Google Earth Engine with MODIS/006/MCD641 dataset </figcaption>\n",
    "</figure>\n",
    "\n",
    "In Figure 1. I plot the a map of images locations that can be used to predict forest fire using these techniques using real workloads such as that acquired from Google Earth Engine NAIP/DQQQ aerial photos with True Color*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5993e4e8-a4f8-4458-8d64-f82300963ecc",
   "metadata": {},
   "source": [
    "h True Color*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6112ccc-7897-4bfc-974b-18dc401494aa",
   "metadata": {},
   "source": [
    "## Fire Samples\n",
    "I sampled the MODIS burn area (red polygons) by \"human random\" means to simply identify the lattitude and longitude of Fire and NoFire regions. The MODIS burn area are the Red polygons, the cyan pins are my intended sample locations.\n",
    "\n",
    "<figure>\n",
    "<img src=\"assets/CA Burn areas 2018 to 2021 sampled.png\", width=\"700\">\n",
    "<figcaption align = \"center\"> Figure 2. Sampled known fire locations used: Google Earth Engine with MODIS/006/MCD64A1 dataset </figcaption>\n",
    "</figure>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5b5d885-030f-4a5c-b426-7af7c85bd37f",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "## NoFire Samples\n",
    "\n",
    "I sampled the MODIS NON burn area (NOT the red polygons) by \"human random\" means. The MODIS burn area are the Red polygons, the darker blue/teal pins are my intended sample locations\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aad74b97-7830-4570-a80d-3ddde0cb9c75",
   "metadata": {},
   "source": [
    "<figure>\n",
    "<img src=\"assets/CA No Burn areas 2018 to 2020 sampled.png\" width=\"500\">\n",
    "<figcaption align = \"center\"> Figure 3. Sampled non fire locations used: Google Earth Engine with MODIS/006/MCD64A1 dataset </figcaption>\n",
    "</figure>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89c2de7b-6c01-4210-baf6-913f06052871",
   "metadata": {},
   "source": [
    "# Use MODIS Burn Area related polygons \n",
    "\n",
    "Hand drawn polygons approximating the MODIS fire region for Northern California from 2016 through 2017"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "547622a1-ff56-434b-b5cb-4708c382cbd1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "ModisOutlineParadise = np.load('data/ModisOutlineParadise.npy')\n",
    "ModisOutlineNorth = np.load('data/ModisOutlineNorth.npy')\n",
    "ModisOutlineWest = np.load('data/ModisOutlineWest.npy' )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bb8a251-69ac-4af0-a7d2-10eeb44f006f",
   "metadata": {},
   "source": [
    "### Function to test whether or no a given latitude, longitude is inside or outside the specified polygon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3445d75-4818-45cf-8963-d3e2c1dd86d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "def point_in_polygon(point, polygon):\n",
    "    x, y = point\n",
    "    n = len(polygon)\n",
    "    inside = False\n",
    "\n",
    "    p1x, p1y = polygon[0]\n",
    "    for i in range(n + 1):\n",
    "        p2x, p2y = polygon[i % n]\n",
    "        if y > min(p1y, p2y):\n",
    "            if y <= max(p1y, p2y):\n",
    "                if x <= max(p1x, p2x):\n",
    "                    if p1y != p2y:\n",
    "                        xints = (y - p1y) * (p2x - p1x) / (p2y - p1y) + p1x\n",
    "                    if p1x == p2x or x <= xints:\n",
    "                        inside = not inside\n",
    "        p1x, p1y = p2x, p2y\n",
    "\n",
    "    return inside"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bac9c74e-8a66-488f-9ff1-f31e6be9a07b",
   "metadata": {},
   "source": [
    "## MODIS Regions Ideal Sampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2b75754-2e7c-42a0-bc9e-8042efec5dba",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.patches import Polygon\n",
    "from shapely.geometry import Polygon as GPoly\n",
    "\n",
    "N_rnd = 600\n",
    "np.random.seed(42)\n",
    "\n",
    "polyP = Polygon(ModisOutlineParadise, closed=True,  fc=(1,0,0,0.3),  ec=(0,0,1,0.3))\n",
    "polyW = Polygon(ModisOutlineWest,     closed=True,  fc=(1,0,0,0.3), ec=(0,0,1,0.3))\n",
    "polyN = Polygon(ModisOutlineNorth,    closed=True,  fc=(1,0,0,0.3), ec=(0,0,1,0.3))\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "ax.add_patch(polyP)\n",
    "ax.add_patch(polyN)\n",
    "ax.add_patch(polyW)\n",
    "\n",
    "# Plot the line segments joining the vertices\n",
    "\n",
    "xp, yp = zip(*ModisOutlineParadise)\n",
    "xw, yw = zip(*ModisOutlineWest)\n",
    "xn, yn = zip(*ModisOutlineNorth)\n",
    "xmin = min(min(xp),min(xw),min(xn))\n",
    "xmax = max(max(xp),max(xw),max(xn))\n",
    "\n",
    "ymin = min(min(yp),min(yw),min(yn))\n",
    "ymax = max(max(yp),max(yw),max(yn))\n",
    "\n",
    "xrnd = 4*np.random.sample(N_rnd) - 124\n",
    "yrnd = 2*np.random.sample(N_rnd) + 39\n",
    "\n",
    "# Color all points blue at first\n",
    "Modis = np.vstack([ModisOutlineWest, ModisOutlineNorth, ModisOutlineParadise])\n",
    "polyWG = GPoly(Modis)\n",
    "prnd = np.array(list(zip(xrnd, yrnd)))\n",
    "plt.scatter(xrnd, yrnd, s= 1, c='b')\n",
    "\n",
    "# Color ModisOutlineWest points red\n",
    "test = []\n",
    "for point in prnd:\n",
    "    test.append(point_in_polygon(point, ModisOutlineWest))\n",
    "test = np.array(test)\n",
    "plt.scatter(xrnd[test==True], yrnd[test==True], s= 1, c='r')\n",
    "\n",
    "# Color ModisOutlineNorth points red\n",
    "test = []\n",
    "for point in prnd:\n",
    "    test.append(point_in_polygon(point, ModisOutlineNorth))\n",
    "test = np.array(test)\n",
    "plt.scatter(xrnd[test==True], yrnd[test==True], s= 1, c='r')\n",
    "\n",
    "# Color ModisOutlineParadise points red\n",
    "test = []\n",
    "for point in prnd:\n",
    "    test.append(point_in_polygon(point, ModisOutlineParadise))\n",
    "test = np.array(test)\n",
    "plt.scatter(xrnd[test==True], yrnd[test==True], s= 1, c='r')\n",
    "\n",
    "\n",
    "# Set the aspect ratio and limits\n",
    "plt.gca().set_aspect('equal')\n",
    "plt.xlim(-124, -120)\n",
    "plt.ylim(39.0, 41.0)\n",
    "plt.grid()\n",
    "# Set labels and title\n",
    "plt.xlabel('Longitude')\n",
    "plt.ylabel('Latitude')\n",
    "plt.title('MODIS Regions Ideal Sampling')\n",
    "#s = [1+10*i for i in range(len(x))]\n",
    "\n",
    "# Show the plot\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "757dedce-e8fc-476f-badc-aa2e982797d2",
   "metadata": {},
   "source": [
    "## MODIS Regions actual Sampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a36f4b75-ca61-42b5-b71e-72fa83b3c74f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.patches import Polygon\n",
    "from shapely.geometry import Polygon as GPoly\n",
    "import random\n",
    "\n",
    "fire = [coord for coord in prnd if point_in_polygon(coord, ModisOutlineParadise)]\n",
    "\n",
    "polyP = Polygon(ModisOutlineParadise, closed=True,  fc=(1,0,0,0.3),  ec=(0,0,1,0.3))\n",
    "polyW = Polygon(ModisOutlineWest,     closed=True,  fc=(1,0,0,0.3), ec=(0,0,1,0.3))\n",
    "polyN = Polygon(ModisOutlineNorth,    closed=True,  fc=(1,0,0,0.3), ec=(0,0,1,0.3))\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "ax.add_patch(polyP)\n",
    "ax.add_patch(polyN)\n",
    "ax.add_patch(polyW)\n",
    "\n",
    "NumFireSample = 0\n",
    "fireP = [coord for coord in prnd if point_in_polygon(coord, ModisOutlineParadise)]\n",
    "x,y = list(zip(*fireP))\n",
    "print(\"Number in Paradise\", len(fireP))\n",
    "plt.scatter(x, y, s= 1, c='r')\n",
    "NumFireSample += len(fireP)\n",
    "\n",
    "fireW = [coord for coord in prnd if point_in_polygon(coord, ModisOutlineWest)]\n",
    "x,y = list(zip(*fireW))\n",
    "print(\"Number in West\", len(fireW))\n",
    "NumFireSample += len(fireW)\n",
    "plt.scatter(x, y, s= 1, c='r')\n",
    "\n",
    "fireN = [coord for coord in prnd if point_in_polygon(coord, ModisOutlineNorth)]\n",
    "x,y = list(zip(*fireN))\n",
    "print(\"Number in North\", len(fireN))\n",
    "NumFireSample += len(fireN)\n",
    "plt.scatter(x, y, s= 1, c='r')\n",
    "print(\"Number of Fire Samples\", NumFireSample)\n",
    "\n",
    "nofire = [coord for coord in prnd if \n",
    "                 (not point_in_polygon(coord, ModisOutlineParadise) \n",
    "                  and not point_in_polygon(coord, ModisOutlineWest) \n",
    "                  and not point_in_polygon(coord, ModisOutlineNorth))]\n",
    "\n",
    "nofireSample = random.sample(nofire, NumFireSample)\n",
    "#x,y = list(zip(*nofire))\n",
    "x,y = list(zip(*nofireSample))\n",
    "plt.scatter(x, y, s= 1, c='b')\n",
    "\n",
    "\n",
    "# Set the aspect ratio and limits\n",
    "plt.gca().set_aspect('equal')\n",
    "plt.xlim(-124, -120)\n",
    "plt.ylim(39.0, 41.0)\n",
    "plt.grid()\n",
    "# Set labels and title\n",
    "plt.xlabel('Longitude')\n",
    "plt.ylabel('Latitude')\n",
    "plt.title('MODIS Regions actual Sampling')\n",
    "\n",
    "# Show the plot\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "696b087d-a053-460d-9d93-d838e04b9cab",
   "metadata": {},
   "source": [
    "## Notices and Disclaimers\n",
    "\n",
    "Intel technologies may require enabled hardware, software or service activation.\n",
    "\n",
    "No product or component can be absolutely secure. \n",
    "\n",
    "Your costs and results may vary. \n",
    "\n",
    "© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}