ch04/ch04.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LqHcEdwqMq2O"
   },
   "source": [
    "# 머신 러닝 교과서 3판"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LYZKU8x4Mq2S"
   },
   "source": [
    "# 4장 - 좋은 훈련 데이터셋 만들기 – 데이터 전처리"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XB3DHSyBMq2S"
   },
   "source": [
    "**아래 링크를 통해 이 노트북을 주피터 노트북 뷰어(nbviewer.jupyter.org)로 보거나 구글 코랩(colab.research.google.com)에서 실행할 수 있습니다.**\n",
    "\n",
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://nbviewer.jupyter.org/github/rickiepark/python-machine-learning-book-3rd-edition/blob/master/ch04/ch04.ipynb\"><img src=\"https://jupyter.org/assets/main-logo.svg\" width=\"28\" />주피터 노트북 뷰어로 보기</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/rickiepark/python-machine-learning-book-3rd-edition/blob/master/ch04/ch04.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />구글 코랩(Colab)에서 실행하기</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GQ9y0Yh0Mq2T"
   },
   "source": [
    "### 목차"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cOhlmn5QMq2T"
   },
   "source": [
    "- 누락된 데이터 다루기\n",
    "  - 테이블 형태 데이터에서 누락된 값 식별\n",
    "  - 누락된 값이 있는 훈련 샘플이나 특성 제외\n",
    "  - 누락된 값 대체\n",
    "  - 사이킷런 추정기 API 익히기\n",
    "- 범주형 데이터 다루기\n",
    "  - 판다스를 사용한 범주형 데이터 인코딩\n",
    "  - 순서가 있는 특성 매핑\n",
    "  - 클래스 레이블 인코딩\n",
    "  - 순서가 없는 특성에 원-핫 인코딩 적용\n",
    "- 데이터셋을 훈련 데이터셋과 테스트 데이터셋으로 나누기\n",
    "- 특성 스케일 맞추기\n",
    "- 유용한 특성 선택\n",
    "  - 모델 복잡도 제한을 위한 L1 규제와 L 2 규제\n",
    "  - L 2 규제의 기하학적 해석\n",
    "  - L1 규제를 사용한 희소성\n",
    "  - 순차 특성 선택 알고리즘\n",
    "- 랜덤 포레스트의 특성 중요도 사용\n",
    "- 요약"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OnTdilY1Mq2T"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Y3pW4prrMq2T",
    "outputId": "272c6503-34dc-4822-cc52-6601450f78d8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already up-to-date: scikit-learn in /usr/local/lib/python3.6/dist-packages (0.24.1)\n",
      "Requirement already satisfied, skipping upgrade: numpy>=1.13.3 in /usr/local/lib/python3.6/dist-packages (from scikit-learn) (1.19.5)\n",
      "Requirement already satisfied, skipping upgrade: scipy>=0.19.1 in /usr/local/lib/python3.6/dist-packages (from scikit-learn) (1.4.1)\n",
      "Requirement already satisfied, skipping upgrade: threadpoolctl>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from scikit-learn) (2.1.0)\n",
      "Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn) (1.0.0)\n"
     ]
    }
   ],
   "source": [
    "# 코랩에서 실행할 경우 최신 버전의 사이킷런을 설치합니다.\n",
    "!pip install --upgrade scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "jmAsiz_rMq2U"
   },
   "outputs": [],
   "source": [
    "from IPython.display import Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H_0FCAEgMq2U"
   },
   "source": [
    "# 누락된 데이터 다루기"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Wx-Z8r-4Mq2U"
   },
   "source": [
    "## 테이블 형태 데이터에서 누락된 값 식별"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "-6aHhkFcMq2U",
    "outputId": "fedff975-74bf-4293-a4f2-6b0888dee31b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C    D\n",
       "0   1.0   2.0   3.0  4.0\n",
       "1   5.0   6.0   NaN  8.0\n",
       "2  10.0  11.0  12.0  NaN"
      ]
     },
     "execution_count": 3,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from io import StringIO\n",
    "import sys\n",
    "\n",
    "csv_data = \\\n",
    "'''A,B,C,D\n",
    "1.0,2.0,3.0,4.0\n",
    "5.0,6.0,,8.0\n",
    "10.0,11.0,12.0,'''\n",
    "\n",
    "# 파이썬 2.7을 사용하는 경우\n",
    "# 다음과 같이 문자열을 유니코드로 변환해야 합니다:\n",
    "\n",
    "if (sys.version_info < (3, 0)):\n",
    "    csv_data = unicode(csv_data)\n",
    "\n",
    "df = pd.read_csv(StringIO(csv_data))\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8aB0B8znMq2V",
    "outputId": "bbd4ec99-73d3-4b95-9c9f-0ce6edffeebd"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "A    0\n",
       "B    0\n",
       "C    1\n",
       "D    1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Go3uwnL6Mq2V",
    "outputId": "f0e3cb70-6605-4781-dffb-171a57486387"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  2.,  3.,  4.],\n",
       "       [ 5.,  6., nan,  8.],\n",
       "       [10., 11., 12., nan]])"
      ]
     },
     "execution_count": 5,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# `values` 속성으로 넘파이 배열을 얻을 수 있습니다\n",
    "df.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t10f63_0Mq2W"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jy1IYwptMq2W"
   },
   "source": [
    "## 누락된 값이 있는 훈련 샘플이나 특성 제외"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 78
    },
    "id": "5vIDbgrwMq2W",
    "outputId": "82fcfeb7-5b7d-4843-e7d2-b26c28b37b72"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A    B    C    D\n",
       "0  1.0  2.0  3.0  4.0"
      ]
     },
     "execution_count": 6,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 누락된 값이 있는 행을 삭제합니다\n",
    "\n",
    "df.dropna(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "fmw16bTYMq2W",
    "outputId": "b3392d54-1af2-4bb5-f4ba-60445a98b91b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B\n",
       "0   1.0   2.0\n",
       "1   5.0   6.0\n",
       "2  10.0  11.0"
      ]
     },
     "execution_count": 7,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 누락된 값이 있는 열을 삭제합니다\n",
    "\n",
    "df.dropna(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "80LOdNzVMq2W",
    "outputId": "743b9c5e-0bed-413a-fb97-8a8bfa506328"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C    D\n",
       "0   1.0   2.0   3.0  4.0\n",
       "1   5.0   6.0   NaN  8.0\n",
       "2  10.0  11.0  12.0  NaN"
      ]
     },
     "execution_count": 8,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 모든 열이 NaN인 행을 삭제합니다\n",
    "\n",
    "df.dropna(how='all')  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 78
    },
    "id": "9Zh9ltQVMq2X",
    "outputId": "2e6e7605-8489-4391-c214-c855d3905985"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     A    B    C    D\n",
       "0  1.0  2.0  3.0  4.0"
      ]
     },
     "execution_count": 9,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# NaN 값이 네 개보다 작은 행을 삭제합니다\n",
    "\n",
    "df.dropna(thresh=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 108
    },
    "id": "0WJCLAekMq2X",
    "outputId": "83ecbf52-2847-4299-94ec-fb4c28b74b3c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C    D\n",
       "0   1.0   2.0   3.0  4.0\n",
       "2  10.0  11.0  12.0  NaN"
      ]
     },
     "execution_count": 10,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 특정 열에 NaN이 있는 행만 삭제합니다(여기서는 'C'열)\n",
    "\n",
    "df.dropna(subset=['C'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RVxHUIGVMq2X"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gCQFUzZWMq2X"
   },
   "source": [
    "## 누락된 값 대체"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7wzvl8t9Mq2X",
    "outputId": "18b264ad-b983-4bfc-d7a2-4b1e63227002",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  2.,  3.,  4.],\n",
       "       [ 5.,  6., nan,  8.],\n",
       "       [10., 11., 12., nan]])"
      ]
     },
     "execution_count": 11,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 원래 배열\n",
    "df.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WSFeOTsHMq2X",
    "outputId": "a0303e06-1b56-47b8-f689-5fdf3d461af0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1. ,  2. ,  3. ,  4. ],\n",
       "       [ 5. ,  6. ,  7.5,  8. ],\n",
       "       [10. , 11. , 12. ,  6. ]])"
      ]
     },
     "execution_count": 12,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 행의 평균으로 누락된 값 대체하기\n",
    "\n",
    "from sklearn.impute import SimpleImputer\n",
    "import numpy as np\n",
    "\n",
    "imr = SimpleImputer(missing_values=np.nan, strategy='mean')\n",
    "imr = imr.fit(df.values)\n",
    "imputed_data = imr.transform(df.values)\n",
    "imputed_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YuRLsdyiMq2Y",
    "outputId": "8ee3a628-7469-4d61-8a36-ce3c5f931618"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.        ,  2.        ,  3.        ,  4.        ],\n",
       "       [ 5.        ,  6.        ,  6.33333333,  8.        ],\n",
       "       [10.        , 11.        , 12.        , 11.        ]])"
      ]
     },
     "execution_count": 13,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import FunctionTransformer\n",
    "\n",
    "ftr_imr = FunctionTransformer(lambda X: imr.fit_transform(X.T).T, validate=False)\n",
    "imputed_data = ftr_imr.fit_transform(df.values)\n",
    "imputed_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2RCBFeEkMq2Y"
   },
   "source": [
    "`SimpleImputer` 클래스의 `add_indicator` 매개변수를 `True`로 지정하면 `indicator_` 속성이 추가되고 `transform()` 메서드가 누락된 값의 위치를 포함된 배열을 반환합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "P_QdmhS-Mq2Y",
    "outputId": "3e8f463d-67e7-43aa-d31a-df3cd2b5cb5e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1. ,  2. ,  3. ,  4. ,  0. ,  0. ],\n",
       "       [ 5. ,  6. ,  7.5,  8. ,  1. ,  0. ],\n",
       "       [10. , 11. , 12. ,  6. ,  0. ,  1. ]])"
      ]
     },
     "execution_count": 14,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imr = SimpleImputer(add_indicator=True)\n",
    "imputed_data = imr.fit_transform(df.values)\n",
    "imputed_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "q_j0xUyNN9WY",
    "outputId": "bb25d3e0-b508-4619-e835-9a3cf27c62a2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 15,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imr.indicator_._precomputed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GdPdZ482Mq2Y"
   },
   "source": [
    "추가된 `indicator_` 속성은 `MissingIndicator` 클래스의 객체입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JLVqXIVkMq2Y",
    "outputId": "a707de91-9ffb-45b7-f56b-0b223f7b5836",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MissingIndicator(error_on_new=False)"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imr.indicator_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ueN0CnpjMq2Y"
   },
   "source": [
    "`MissingIndicator` 객체의 `features_` 속성은 누락된 값이 있는 특성의 인덱스를 담고 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "cdiUcNKiMq2Z",
    "outputId": "0e78a69b-78dd-4d8d-a52a-b05de4473af9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 3])"
      ]
     },
     "execution_count": 17,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imr.indicator_.features_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pfMmBdGlMq2Z"
   },
   "source": [
    "`MissingIndicator` 객체의 `fit_transform()` 메서드를 호출하면 `features_` 속성에 담긴 특성에서 누락된 값의 위치를 나타내는 배열을 반환합니다. 여기에서는 세 번째, 네 번째 특성의 누락된 값의 위치입니다. 이 배열의 값이 `SimpleImputer` 객체의 `transfrom()` 메서드가 반환한 배열의 마지막 두 열에 해당합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "FCVHbIJjMq2Z",
    "outputId": "0f39de25-f9c2-47e4-aef8-fa83508dc634"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[False, False],\n",
       "       [ True, False],\n",
       "       [False,  True]])"
      ]
     },
     "execution_count": 18,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imr.indicator_.fit_transform(df.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bka20kH5Mq2Z",
    "outputId": "bcd27de0-57bb-466f-f5e3-e9439d2d0f60",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.,  2.,  3.,  4.],\n",
       "       [ 5.,  6., nan,  8.],\n",
       "       [10., 11., 12., nan]])"
      ]
     },
     "execution_count": 19,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 사이킷런 0.24 버전에서 추가되었습니다.\n",
    "imr.inverse_transform(imputed_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VLIH95sZMq2Z"
   },
   "source": [
    "`SimpleImputer`는 한 특성의 통곗값을 사용하여 누락된 값을 채웁니다. 이와 달리 `IterativeImputer` 클래스는 다른 특성을 사용하여 누락된 값을 예측합니다. 먼저 `initial_strategy` 매개변수에 지정된 방식으로 누락된 값을 초기화합니다. 그다음 누락된 값이 있는 한 특성을 타깃으로 삼고 다른 특성을 사용해 모델을 훈련하여 예측합니다. 이런 식으로 누락된 값이 있는 모든 특성을 순회합니다.\n",
    "\n",
    "`initial_strategy` 매개변수에 지정할 수 있는 값은 `SimpleImputer`와 동일하게 `'mean'`, `'median'`, `'most_frequent'`, `'constant'`가 가능합니다.\n",
    "\n",
    "예측할 특성을 선택하는 순서는 누락된 값이 가장 적은 특성부터 선택하는 `'ascending'`, 누락된 값이 가장 큰 특성부터 선택하는 `'descending'`, 왼쪽에서 오른쪽으로 선택하는 `'roman'`, 오른쪽에서 왼쪽으로 선택하는 `'arabic'`, 랜덤하게 고르는 `'random'`이 있습니다. 기본값은 `'ascending'`입니다.\n",
    "\n",
    "특성 예측은 종료 조건을 만족할 때까지 반복합니다. 각 반복 단계에서 이전 단계와 절댓값 차이 중 가장 큰 값이 누락된 값을 제외하고 가장 큰 절댓값에 `tol` 매개변수를 곱한 것보다 작을 경우 종료합니다. `tol` 매개변수 기본값은 1e-3입니다. 또는 `max_iter` 매개변수에서 지정한 횟수에 도달할 때 종료합니다. `max_iter`의 기본값은 10입니다.\n",
    "\n",
    "예측에 사용하는 모델은 `estimator` 매개변수에서 지정할 수 있으며 기본적으로 `BayesianRidge` 클래스를 사용합니다. 예측에 사용할 특성 개수는 `n_nearest_features`에서 지정할 수 있으며 상관 계수가 높은 특성을 우선하여 랜덤하게 선택합니다. 기본값은 `None`으로 모든 특성을 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "iuTQp_czMq2Z",
    "outputId": "01935bee-7f43-4ac9-c5b9-0216684a73d7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.        ,  2.        ,  3.        ,  4.        ],\n",
       "       [ 5.        ,  6.        ,  7.00047063,  8.        ],\n",
       "       [10.        , 11.        , 12.        , 12.99964527]])"
      ]
     },
     "execution_count": 20,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.experimental import enable_iterative_imputer\n",
    "from sklearn.impute import IterativeImputer\n",
    "\n",
    "iimr = IterativeImputer()\n",
    "iimr.fit_transform(df.values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lDqqS971Mq2a"
   },
   "source": [
    "`KNNImputer` 클래스는 K-최근접 이웃 방법을 사용해 누락된 값을 채웁니다. 최근접 이웃의 개수는 `n_neighbors` 매개변수로 지정하며 기본값은 5입니다. 샘플 개수가 `n_neighbors` 보다 작으면 `SimpleImputer(strategy='mean')`과 결과가 같습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hyPQKItLMq2a",
    "outputId": "5996a6b2-a899-43cb-8a48-7890335df99e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1. ,  2. ,  3. ,  4. ],\n",
       "       [ 5. ,  6. ,  7.5,  8. ],\n",
       "       [10. , 11. , 12. ,  6. ]])"
      ]
     },
     "execution_count": 21,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.impute import KNNImputer\n",
    "\n",
    "kimr = KNNImputer()\n",
    "kimr.fit_transform(df.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "3VYSAQbNMq2a",
    "outputId": "32033061-73fc-4db6-a627-667866342162",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>7.5</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C    D\n",
       "0   1.0   2.0   3.0  4.0\n",
       "1   5.0   6.0   7.5  8.0\n",
       "2  10.0  11.0  12.0  6.0"
      ]
     },
     "execution_count": 22,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.fillna(df.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JN7-WTpsMq2a"
   },
   "source": [
    "`fillna()` 메서드의 `method` 매개변수를 사용하여 누락된 값을 채울 수도 있습니다. `bfill` 또는 `backfill`은 누락된 값을 다음 행의 값으로 채웁니다. `ffill` 또는 `pad`는 누락된 값을 이전 행의 값으로 채웁니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "o9uFQkZHMq2a",
    "outputId": "84ccfefc-e146-4c31-a26c-fc73abcc75f9",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C    D\n",
       "0   1.0   2.0   3.0  4.0\n",
       "1   5.0   6.0  12.0  8.0\n",
       "2  10.0  11.0  12.0  NaN"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.fillna(method='bfill') # method='backfill'와 같습니다"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "WGw6iY8pMq2a",
    "outputId": "ee1350c4-9d47-4b4b-e73b-08bf58c0c111"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C    D\n",
       "0   1.0   2.0   3.0  4.0\n",
       "1   5.0   6.0   3.0  8.0\n",
       "2  10.0  11.0  12.0  8.0"
      ]
     },
     "execution_count": 24,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.fillna(method='ffill') # method='pad'와 같습니다"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AtatT0P8Mq2b"
   },
   "source": [
    "`axis=1`로 지정하면 행이 아니라 열을 사용합니다. 예를 들어 다음처럼 지정하면 이전 열의 값으로 누락된 값을 채웁니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "zDA7-B5mMq2b",
    "outputId": "ecf017d1-0eb9-4d36-8b96-14471c8c7e13"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>12.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      A     B     C     D\n",
       "0   1.0   2.0   3.0   4.0\n",
       "1   5.0   6.0   6.0   8.0\n",
       "2  10.0  11.0  12.0  12.0"
      ]
     },
     "execution_count": 25,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.fillna(method='ffill', axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qnfELrGfMq2b"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bbiT9PxQMq2b"
   },
   "source": [
    "## 사이킷런 추정기 API 익히기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 275
    },
    "id": "nMten7quMq2b",
    "outputId": "c8f026b8-61bd-4530-a984-1e49091784f6"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://git.io/Jtmwv\" width=\"400\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 26,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='https://git.io/Jtmwv', width=400) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 369
    },
    "id": "sOC87DMPMq2b",
    "outputId": "1d3ddea5-ca53-4ebd-c95b-7183badf7fd0"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://git.io/JtYZW\" width=\"300\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 27,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='https://git.io/JtYZW', width=300) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zEvoptE7Mq2c"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5nUCfeN0Mq2c"
   },
   "source": [
    "# 범주형 데이터 다루기"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iPxH62giMq2c"
   },
   "source": [
    "## 판다스를 사용한 범주형 데이터 인코딩"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "dPoi0v6eMq2c",
    "outputId": "2cf0b5b3-4ebb-44c1-b0a5-a5fed58ffb39"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>M</td>\n",
       "      <td>10.1</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>L</td>\n",
       "      <td>13.5</td>\n",
       "      <td>class1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>XL</td>\n",
       "      <td>15.3</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color size  price classlabel\n",
       "0  green    M   10.1     class2\n",
       "1    red    L   13.5     class1\n",
       "2   blue   XL   15.3     class2"
      ]
     },
     "execution_count": 28,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.DataFrame([['green', 'M', 10.1, 'class2'],\n",
    "                   ['red', 'L', 13.5, 'class1'],\n",
    "                   ['blue', 'XL', 15.3, 'class2']])\n",
    "\n",
    "df.columns = ['color', 'size', 'price', 'classlabel']\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ECbA0qITMq2c"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M4ytrqnUMq2c"
   },
   "source": [
    "## 순서가 있는 특성 매핑"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "JGKB8nr3Mq2c",
    "outputId": "e72f305a-a6b3-4671-9611-93c6599d2e78"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>1</td>\n",
       "      <td>10.1</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>2</td>\n",
       "      <td>13.5</td>\n",
       "      <td>class1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>3</td>\n",
       "      <td>15.3</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color  size  price classlabel\n",
       "0  green     1   10.1     class2\n",
       "1    red     2   13.5     class1\n",
       "2   blue     3   15.3     class2"
      ]
     },
     "execution_count": 29,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "size_mapping = {'XL': 3,\n",
    "                'L': 2,\n",
    "                'M': 1}\n",
    "\n",
    "df['size'] = df['size'].map(size_mapping)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tXndUtR3Mq2c",
    "outputId": "14866808-f281-4eae-9b50-6d1eaa21a855"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     M\n",
       "1     L\n",
       "2    XL\n",
       "Name: size, dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inv_size_mapping = {v: k for k, v in size_mapping.items()}\n",
    "df['size'].map(inv_size_mapping)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r3oztwD7Mq2d"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OAbFJUkUMq2d"
   },
   "source": [
    "## 클래스 레이블 인코딩"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SDHpWWn0Mq2d",
    "outputId": "0786ff15-8746-4191-f207-26903b793971"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'class1': 0, 'class2': 1}"
      ]
     },
     "execution_count": 31,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# 클래스 레이블을 문자열에서 정수로 바꾸기 위해\n",
    "# 매핑 딕셔너리를 만듭니다\n",
    "class_mapping = {label: idx for idx, label in enumerate(np.unique(df['classlabel']))}\n",
    "class_mapping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "zLJHGfvmMq2d",
    "outputId": "2d110a98-135d-4449-eaa8-61c644d148a1"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>1</td>\n",
       "      <td>10.1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>2</td>\n",
       "      <td>13.5</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>3</td>\n",
       "      <td>15.3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color  size  price  classlabel\n",
       "0  green     1   10.1           1\n",
       "1    red     2   13.5           0\n",
       "2   blue     3   15.3           1"
      ]
     },
     "execution_count": 32,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 클래스 레이블을 문자열에서 정수로 바꿉니다\n",
    "df['classlabel'] = df['classlabel'].map(class_mapping)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "kKUp4itGMq2d",
    "outputId": "54287df0-ded2-48d4-a463-4a494e3b654f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>1</td>\n",
       "      <td>10.1</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>2</td>\n",
       "      <td>13.5</td>\n",
       "      <td>class1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>3</td>\n",
       "      <td>15.3</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color  size  price classlabel\n",
       "0  green     1   10.1     class2\n",
       "1    red     2   13.5     class1\n",
       "2   blue     3   15.3     class2"
      ]
     },
     "execution_count": 33,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 클래스 레이블을 거꾸로 매핑합니다\n",
    "inv_class_mapping = {v: k for k, v in class_mapping.items()}\n",
    "df['classlabel'] = df['classlabel'].map(inv_class_mapping)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rSKIVnixMq2d",
    "outputId": "98a12e57-2de2-475d-cca5-9a9ec4da9c67"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 1])"
      ]
     },
     "execution_count": 34,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# 사이킷런의 LabelEncoder을 사용한 레이블 인코딩\n",
    "class_le = LabelEncoder()\n",
    "y = class_le.fit_transform(df['classlabel'].values)\n",
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "EYxisQriMq2d",
    "outputId": "17492d5b-dc16-4821-da72-cd86e97ae53c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['class2', 'class1', 'class2'], dtype=object)"
      ]
     },
     "execution_count": 35,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 거꾸로 매핑\n",
    "class_le.inverse_transform(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9Or2fRLLMq2e",
    "outputId": "cae130be-892f-45a8-9785-667c4880216b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['class1', 'class2'], dtype=object)"
      ]
     },
     "execution_count": 36,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class_le.classes_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pIIuEd5XMq2e"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5wqnjFLgMq2e"
   },
   "source": [
    "## 순서가 없는 특성에 원-핫 인코딩 적용"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7Zpy1Xb7Mq2e",
    "outputId": "4e6965df-c840-4fde-c208-f13134cbdaf7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 1, 10.1],\n",
       "       [2, 2, 13.5],\n",
       "       [0, 3, 15.3]], dtype=object)"
      ]
     },
     "execution_count": 37,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = df[['color', 'size', 'price']].values\n",
    "color_le = LabelEncoder()\n",
    "X[:, 0] = color_le.fit_transform(X[:, 0])\n",
    "X\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RnvNkt2tMq2e"
   },
   "source": [
    "`OrdinalEncoder`와 `ColumnTransformer` 함께 사용하기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XAPjHy1lMq2e",
    "outputId": "df292774-0178-4096-c624-9c5976016716"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1],\n",
       "       [2],\n",
       "       [0]])"
      ]
     },
     "execution_count": 38,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.preprocessing import OrdinalEncoder\n",
    "\n",
    "ord_enc = OrdinalEncoder(dtype=np.int)\n",
    "col_trans = ColumnTransformer([('ord_enc', ord_enc, ['color'])])\n",
    "X_trans = col_trans.fit_transform(df)\n",
    "X_trans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KyQpw3VyMq2e",
    "outputId": "b0011f90-15e9-486e-9af7-599eb6545e95"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([['green'],\n",
       "       ['red'],\n",
       "       ['blue']], dtype=object)"
      ]
     },
     "execution_count": 39,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col_trans.named_transformers_['ord_enc'].inverse_transform(X_trans)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "To3Hh2qCMq2f",
    "outputId": "6c9d373e-8cef-446d-deec-bb4921ddcd23"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 1., 0.],\n",
       "       [0., 0., 1.],\n",
       "       [1., 0., 0.]])"
      ]
     },
     "execution_count": 40,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder\n",
    "\n",
    "X = df[['color', 'size', 'price']].values\n",
    "color_ohe = OneHotEncoder()\n",
    "color_ohe.fit_transform(X[:, 0].reshape(-1, 1)).toarray()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6SeKAgdGMq2f",
    "outputId": "24fb7ad8-afc2-434a-c5c5-c480716e2bd1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.0, 1.0, 0.0, 1, 10.1],\n",
       "       [0.0, 0.0, 1.0, 2, 13.5],\n",
       "       [1.0, 0.0, 0.0, 3, 15.3]], dtype=object)"
      ]
     },
     "execution_count": 41,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.compose import ColumnTransformer\n",
    "\n",
    "X = df[['color', 'size', 'price']].values\n",
    "c_transf = ColumnTransformer([ ('onehot', OneHotEncoder(), [0]),\n",
    "                               ('nothing', 'passthrough', [1, 2])])\n",
    "c_transf.fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qCb4zK_nMq2f"
   },
   "source": [
    "`OneHotEncoder`의 `dtype` 매개변수를 `np.int`로 지정하여 정수로 원-핫 인코딩합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gou8GnFYMq2f",
    "outputId": "e064308c-b26a-4d57-bb79-bdbfc0728587"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 0, 1, 10.1],\n",
       "       [0, 0, 1, 2, 13.5],\n",
       "       [1, 0, 0, 3, 15.3]], dtype=object)"
      ]
     },
     "execution_count": 42,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c_transf = ColumnTransformer([ ('onehot', OneHotEncoder(dtype=np.int), [0]),\n",
    "                               ('nothing', 'passthrough', [1, 2])])\n",
    "c_transf.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "T9Pq7e8IMq2g",
    "outputId": "dcb77c36-ee71-4f51-db31-3f697393d1ca"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>size</th>\n",
       "      <th>color_blue</th>\n",
       "      <th>color_green</th>\n",
       "      <th>color_red</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13.5</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15.3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   price  size  color_blue  color_green  color_red\n",
       "0   10.1     1           0            1          0\n",
       "1   13.5     2           0            0          1\n",
       "2   15.3     3           1            0          0"
      ]
     },
     "execution_count": 43,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 원-핫 인코딩 via 판다스\n",
    "\n",
    "pd.get_dummies(df[['price', 'color', 'size']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4NQ8um5WMq2g"
   },
   "source": [
    "`column` 매개변수를 사용하여 변환하려는 특성을 구체적으로 지정할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "VLafyvqcMq2g",
    "outputId": "f71332b3-1d91-47fa-fdc7-5cd6f8fd01c2"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>color</th>\n",
       "      <th>size_1</th>\n",
       "      <th>size_2</th>\n",
       "      <th>size_3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.1</td>\n",
       "      <td>green</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13.5</td>\n",
       "      <td>red</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15.3</td>\n",
       "      <td>blue</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   price  color  size_1  size_2  size_3\n",
       "0   10.1  green       1       0       0\n",
       "1   13.5    red       0       1       0\n",
       "2   15.3   blue       0       0       1"
      ]
     },
     "execution_count": 44,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.get_dummies(df[['price', 'color', 'size']], columns=['size'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "vJxTK0oiMq2g",
    "outputId": "0b7bee9e-77dc-4318-cfc1-860a94a3cc4b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>price</th>\n",
       "      <th>size</th>\n",
       "      <th>color_green</th>\n",
       "      <th>color_red</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10.1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13.5</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15.3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   price  size  color_green  color_red\n",
       "0   10.1     1            1          0\n",
       "1   13.5     2            0          1\n",
       "2   15.3     3            0          0"
      ]
     },
     "execution_count": 45,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get_dummies에서 다중 공선성 문제 처리\n",
    "\n",
    "pd.get_dummies(df[['price', 'color', 'size']], drop_first=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CY1QY_smMq2g",
    "outputId": "db07964d-89ba-4232-a077-842d51af5206"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.0, 0.0, 1, 10.1],\n",
       "       [0.0, 1.0, 2, 13.5],\n",
       "       [0.0, 0.0, 3, 15.3]], dtype=object)"
      ]
     },
     "execution_count": 46,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# OneHotEncoder에서 다중 공선성 문제 처리\n",
    "\n",
    "color_ohe = OneHotEncoder(categories='auto', drop='first')\n",
    "c_transf = ColumnTransformer([ ('onehot', color_ohe, [0]),\n",
    "                               ('nothing', 'passthrough', [1, 2])])\n",
    "c_transf.fit_transform(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ICXBFMm8Mq2g"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u_yxqE6MMq2h"
   },
   "source": [
    "## 추가 내용: 순서가 있는 특성 인코딩"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OTD3jv9KMq2h"
   },
   "source": [
    "순서가 있는 특성의 범주 사이에서 수치적 크기에 대해 확신이 없거나 두 범주 사이의 순서를 정의할 수 없다면 임곗값을 사용해 0/1로 인코딩할 수 있습니다. 예를 들어 M, L, XL 값을 가진 특성 `size`를 2개의 새로운 특성 \"x > M\"와 \"x > L\"로 나눌 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "9CvGeqsXMq2h",
    "outputId": "43a2c67d-20a8-442d-9136-ecdb8e52ca4e"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>size</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>M</td>\n",
       "      <td>10.1</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>L</td>\n",
       "      <td>13.5</td>\n",
       "      <td>class1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>XL</td>\n",
       "      <td>15.3</td>\n",
       "      <td>class2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color size  price classlabel\n",
       "0  green    M   10.1     class2\n",
       "1    red    L   13.5     class1\n",
       "2   blue   XL   15.3     class2"
      ]
     },
     "execution_count": 47,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame([['green', 'M', 10.1, 'class2'],\n",
    "                   ['red', 'L', 13.5, 'class1'],\n",
    "                   ['blue', 'XL', 15.3, 'class2']])\n",
    "\n",
    "df.columns = ['color', 'size', 'price', 'classlabel']\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1DdNScVOMq2h"
   },
   "source": [
    "판다스 데이터프레임의 `apply` 메서드를 사용해 임곗값 기준으로 특성을 인코딩하는 `lambda` 함수를 적용할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 138
    },
    "id": "dOA0I-8RMq2h",
    "outputId": "3af93bee-4434-4ce9-9b50-fb42a22d6873"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>color</th>\n",
       "      <th>price</th>\n",
       "      <th>classlabel</th>\n",
       "      <th>x &gt; M</th>\n",
       "      <th>x &gt; L</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>green</td>\n",
       "      <td>10.1</td>\n",
       "      <td>class2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>red</td>\n",
       "      <td>13.5</td>\n",
       "      <td>class1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>blue</td>\n",
       "      <td>15.3</td>\n",
       "      <td>class2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   color  price classlabel  x > M  x > L\n",
       "0  green   10.1     class2      0      0\n",
       "1    red   13.5     class1      1      0\n",
       "2   blue   15.3     class2      1      1"
      ]
     },
     "execution_count": 48,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['x > M'] = df['size'].apply(lambda x: 1 if x in {'L', 'XL'} else 0)\n",
    "df['x > L'] = df['size'].apply(lambda x: 1 if x == 'XL' else 0)\n",
    "\n",
    "del df['size']\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7UMoj_GpMq2h"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m6mpGl2rMq2h"
   },
   "source": [
    "# 데이터셋을 훈련 데이터셋과 테스트 데이터셋으로 나누기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 270
    },
    "id": "i_v5Th0ZMq2i",
    "outputId": "c8a68c78-239b-42a5-e81c-631b5c17c314"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class labels [1 2 3]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Class label</th>\n",
       "      <th>Alcohol</th>\n",
       "      <th>Malic acid</th>\n",
       "      <th>Ash</th>\n",
       "      <th>Alcalinity of ash</th>\n",
       "      <th>Magnesium</th>\n",
       "      <th>Total phenols</th>\n",
       "      <th>Flavanoids</th>\n",
       "      <th>Nonflavanoid phenols</th>\n",
       "      <th>Proanthocyanins</th>\n",
       "      <th>Color intensity</th>\n",
       "      <th>Hue</th>\n",
       "      <th>OD280/OD315 of diluted wines</th>\n",
       "      <th>Proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>14.23</td>\n",
       "      <td>1.71</td>\n",
       "      <td>2.43</td>\n",
       "      <td>15.6</td>\n",
       "      <td>127</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.06</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2.29</td>\n",
       "      <td>5.64</td>\n",
       "      <td>1.04</td>\n",
       "      <td>3.92</td>\n",
       "      <td>1065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>13.20</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2.14</td>\n",
       "      <td>11.2</td>\n",
       "      <td>100</td>\n",
       "      <td>2.65</td>\n",
       "      <td>2.76</td>\n",
       "      <td>0.26</td>\n",
       "      <td>1.28</td>\n",
       "      <td>4.38</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.40</td>\n",
       "      <td>1050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>13.16</td>\n",
       "      <td>2.36</td>\n",
       "      <td>2.67</td>\n",
       "      <td>18.6</td>\n",
       "      <td>101</td>\n",
       "      <td>2.80</td>\n",
       "      <td>3.24</td>\n",
       "      <td>0.30</td>\n",
       "      <td>2.81</td>\n",
       "      <td>5.68</td>\n",
       "      <td>1.03</td>\n",
       "      <td>3.17</td>\n",
       "      <td>1185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>14.37</td>\n",
       "      <td>1.95</td>\n",
       "      <td>2.50</td>\n",
       "      <td>16.8</td>\n",
       "      <td>113</td>\n",
       "      <td>3.85</td>\n",
       "      <td>3.49</td>\n",
       "      <td>0.24</td>\n",
       "      <td>2.18</td>\n",
       "      <td>7.80</td>\n",
       "      <td>0.86</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>13.24</td>\n",
       "      <td>2.59</td>\n",
       "      <td>2.87</td>\n",
       "      <td>21.0</td>\n",
       "      <td>118</td>\n",
       "      <td>2.80</td>\n",
       "      <td>2.69</td>\n",
       "      <td>0.39</td>\n",
       "      <td>1.82</td>\n",
       "      <td>4.32</td>\n",
       "      <td>1.04</td>\n",
       "      <td>2.93</td>\n",
       "      <td>735</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Class label  Alcohol  ...  OD280/OD315 of diluted wines  Proline\n",
       "0            1    14.23  ...                          3.92     1065\n",
       "1            1    13.20  ...                          3.40     1050\n",
       "2            1    13.16  ...                          3.17     1185\n",
       "3            1    14.37  ...                          3.45     1480\n",
       "4            1    13.24  ...                          2.93      735\n",
       "\n",
       "[5 rows x 14 columns]"
      ]
     },
     "execution_count": 49,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_wine = pd.read_csv('https://archive.ics.uci.edu/'\n",
    "                      'ml/machine-learning-databases/wine/wine.data',\n",
    "                      header=None)\n",
    "\n",
    "# UCI 머신러닝 저장소의 Wine 데이터셋에 접근되지 않을 때\n",
    "# 다음 코드의 주석을 제거하고 로컬 경로에서 데이터셋을 읽으세요:\n",
    "\n",
    "# df_wine = pd.read_csv('wine.data', header=None)\n",
    "\n",
    "\n",
    "df_wine.columns = ['Class label', 'Alcohol', 'Malic acid', 'Ash',\n",
    "                   'Alcalinity of ash', 'Magnesium', 'Total phenols',\n",
    "                   'Flavanoids', 'Nonflavanoid phenols', 'Proanthocyanins',\n",
    "                   'Color intensity', 'Hue', 'OD280/OD315 of diluted wines',\n",
    "                   'Proline']\n",
    "\n",
    "print('Class labels', np.unique(df_wine['Class label']))\n",
    "df_wine.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "id": "fimdvdJPMq2i"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X, y = df_wine.iloc[:, 1:].values, df_wine.iloc[:, 0].values\n",
    "\n",
    "X_train, X_test, y_train, y_test =\\\n",
    "    train_test_split(X, y, \n",
    "                     test_size=0.3, \n",
    "                     random_state=0, \n",
    "                     stratify=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NndbbqcRMq2i"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xJ3QxaZYMq2i"
   },
   "source": [
    "# 특성 스케일 맞추기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "id": "Khy5PTnMMq2i"
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "mms = MinMaxScaler()\n",
    "X_train_norm = mms.fit_transform(X_train)\n",
    "X_test_norm = mms.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "id": "jlb3rF2iMq2i"
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "stdsc = StandardScaler()\n",
    "X_train_std = stdsc.fit_transform(X_train)\n",
    "X_test_std = stdsc.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VsaLJZE2Mq2j"
   },
   "source": [
    "예제:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "C7kJaZkqMq2j",
    "outputId": "a9900bcb-b0a2-4f0a-cca7-2c45747f4652"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "표준화: [-1.46385011 -0.87831007 -0.29277002  0.29277002  0.87831007  1.46385011]\n",
      "정규화: [0.  0.2 0.4 0.6 0.8 1. ]\n"
     ]
    }
   ],
   "source": [
    "ex = np.array([0, 1, 2, 3, 4, 5])\n",
    "\n",
    "print('표준화:', (ex - ex.mean()) / ex.std())\n",
    "\n",
    "# 판다스는 기본적으로 ddof=1를 사용합니다(샘플 표준 편차).\n",
    "# 반면 넘파이 std 메서드와 StandardScaler는 ddof=0를 사용합니다.\n",
    "\n",
    "# 정규화합니다\n",
    "print('정규화:', (ex - ex.min()) / (ex.max() - ex.min()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "id": "AAu1y3x_Mq2j"
   },
   "outputs": [],
   "source": [
    "ex_2d = np.reshape(ex, (-1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "x7VlzH06Mq2j"
   },
   "source": [
    "`RobustScaler`는 중간값(q2) 값을 빼고 1사분위(q1)와 3사분위(q3)의 차이로 나누어 데이터의 스케일을 조정합니다.\n",
    "\n",
    "$ x_{robust}^{(i)} = \\dfrac{x^{(i)} - q_2}{q_3 - q_1}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "id": "gmLmwQqeMq2j"
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import RobustScaler\n",
    "rbs = RobustScaler()\n",
    "X_train_robust = rbs.fit_transform(X_train)\n",
    "X_test_robust = rbs.fit_transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "u6CC1k-NMq2j",
    "outputId": "d320d6dd-0b9a-479b-ecfe-bd9ed5aecd17"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1. , -0.6, -0.2,  0.2,  0.6,  1. ])"
      ]
     },
     "execution_count": 56,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(ex - np.percentile(ex, 50)) / (np.percentile(ex, 75) - np.percentile(ex, 25))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YAT1GxwzMq2j"
   },
   "source": [
    "`MaxAbsScaler`는 각 특성별로 데이터를 최댓 절댓값으로 나눕니다. 따라서 각 특성의 최댓값은 1이 됩니다. 전체 특성은 [-1, 1] 범위로 변경됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "id": "HhQvsTHAMq2k"
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MaxAbsScaler\n",
    "mas = MaxAbsScaler()\n",
    "X_train_maxabs = mas.fit_transform(X_train)\n",
    "X_test_maxabs = mas.fit_transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1OIyaEaDMq2k",
    "outputId": "a7afc18c-d862-4705-f98b-625c496cd792"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0. , 0.2, 0.4, 0.6, 0.8, 1. ])"
      ]
     },
     "execution_count": 58,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ex / np.max(np.abs(ex))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d2N0nJP-Mq2k"
   },
   "source": [
    "`StandardScaler`, `MinMaxScaler`, `RobustScaler`, `MaxAbsScaler`에 대응하는 `scale()`, `minmax_scale()`, `robust_scale()`, `maxabs_scale()` 함수가 있습니다. 이 함수들은 1차원 배열도 입력 받을 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eLy80nX3Mq2k",
    "outputId": "3e46544f-6d4c-4bcf-b96f-e1563606af02"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "StandardScaler: [-1.46385011 -0.87831007 -0.29277002  0.29277002  0.87831007  1.46385011]\n",
      "MinMaxScaler: [0.  0.2 0.4 0.6 0.8 1. ]\n",
      "RobustScaler: [-1.  -0.6 -0.2  0.2  0.6  1. ]\n",
      "MaxAbsScaler: [0.  0.2 0.4 0.6 0.8 1. ]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import scale, minmax_scale, robust_scale, maxabs_scale\n",
    "print('StandardScaler:', scale(ex))\n",
    "print('MinMaxScaler:', minmax_scale(ex))\n",
    "print('RobustScaler:', robust_scale(ex))\n",
    "print('MaxAbsScaler:', maxabs_scale(ex))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oLrCUS3LMq2k"
   },
   "source": [
    "`MaxAbsScaler`, `maxabs_scaler()`는 데이터를 중앙에 맞추지 않기 때문에 희소 행렬을 사용할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "id": "fq473Ek_Mq2k"
   },
   "outputs": [],
   "source": [
    "from scipy import sparse\n",
    "X_train_sparse = sparse.csr_matrix(X_train)\n",
    "\n",
    "X_train_maxabs = mas.fit_transform(X_train_sparse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0W3i4xaUMq2k"
   },
   "source": [
    "`RobustScaler`는 희소 행렬을 사용해 훈련할 수 없지만 변환은 가능합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "id": "dad5Q-3KMq2k"
   },
   "outputs": [],
   "source": [
    "X_train_robust = rbs.transform(X_train_sparse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hB_yqD6-Mq2l"
   },
   "source": [
    "마지막으로 `Normalizer` 클래스와 `normalize()` 함수는 특성이 아니라 샘플별로 정규화를 수행합니다. 또한 희소 행렬도 처리할 수 있습니다. 기본적으로 샘플의 L2 노름이 1이 되도록 정규화합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "id": "zW77xRE4Mq2l"
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import Normalizer\n",
    "\n",
    "nrm = Normalizer()\n",
    "X_train_l2 = nrm.fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XSGDsHVcMq2l"
   },
   "source": [
    "예를 위해서 `ex` 데이터에 특성을 하나 더 추가해 보겠습니다. 0 나눗셈 오류를 다루기 번거로우므로 편의상 0을 제외합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "B-U6YjhzMq2l",
    "outputId": "9d0c3230-ef70-4231-c062-92648611e55d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1,  2,  3,  4,  5],\n",
       "       [ 1,  4,  9, 16, 25]])"
      ]
     },
     "execution_count": 63,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ex_2f = np.vstack((ex[1:], ex[1:]**2))\n",
    "ex_2f"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AMbSirpqMq2l"
   },
   "source": [
    "L2 노름의 공식은 다음과 같습니다.\n",
    "\n",
    "$ \\lVert \\boldsymbol{x} \\rVert_2 = \\sqrt{x_1^2 + x_2^2 + \\cdots + x_n^2} $\n",
    "\n",
    "먼저 샘플 별로 특성의 제곱을 더하기 위해 `axis=1`을 사용합니다. 이 값의 제곱근을 구하면 L2 노름입니다. 그 다음 각 샘플의 특성을 해당 L2 노름으로 나눕니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "E8jbH72UMq2l",
    "outputId": "848b4362-7c37-401a-fc09-ad342f1961d4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 7.41619849 31.28897569]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[0.13483997, 0.26967994, 0.40451992, 0.53935989, 0.67419986],\n",
       "       [0.03196014, 0.12784055, 0.28764125, 0.51136222, 0.79900347]])"
      ]
     },
     "execution_count": 64,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l2_norm = np.sqrt(np.sum(ex_2f ** 2, axis=1))\n",
    "print(l2_norm)\n",
    "ex_2f / l2_norm.reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BaaZNtcpMq2l"
   },
   "source": [
    "`norm='l1'`으로 지정하면 절댓값인 L1 노름을 사용합니다. L1 노름의 공식은 다음과 같습니다.\n",
    "\n",
    "$ \\lVert \\boldsymbol{x} \\rVert_1 = \\lvert x_1 \\rvert + \\lvert x_2 \\rvert + \\cdots + \\lvert x_n \\rvert $\n",
    "\n",
    "앞에서와 마찬가지로 샘플 별로 L1 노름을 계산한 다음 각 샘플을 나눕니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "cPFXrFFBMq2l",
    "outputId": "7b687ad8-a951-45cf-d996-659b7da4ba9e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[15 55]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[0.06666667, 0.13333333, 0.2       , 0.26666667, 0.33333333],\n",
       "       [0.01818182, 0.07272727, 0.16363636, 0.29090909, 0.45454545]])"
      ]
     },
     "execution_count": 65,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l1_norm = np.sum(np.abs(ex_2f), axis=1)\n",
    "print(l1_norm)\n",
    "ex_2f / l1_norm.reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hsmMub2vMq2m"
   },
   "source": [
    "`norm='max'`로 지정하면 최대 절댓값을 사용하여 나눕니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UTj2mst_Mq2m",
    "outputId": "5f660198-a62b-46e1-f826-d51e824da070"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 5 25]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[0.2 , 0.4 , 0.6 , 0.8 , 1.  ],\n",
       "       [0.04, 0.16, 0.36, 0.64, 1.  ]])"
      ]
     },
     "execution_count": 66,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_norm = np.max(np.abs(ex_2f), axis=1)\n",
    "print(max_norm)\n",
    "ex_2f / max_norm.reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Phu1_g_6Mq2m"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8girZgcUMq2m"
   },
   "source": [
    "# 유용한 특성 선택"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oFGd4DhNMq2m"
   },
   "source": [
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0PgT2drCMq2m"
   },
   "source": [
    "## 모델 복잡도 제한을 위한 L1 규제와 L2 규제"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XgMUmyRAMq2m"
   },
   "source": [
    "## L2 규제의 기하학적 해석"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 404
    },
    "id": "ULk6PzD4Mq2m",
    "outputId": "34276084-a996-4191-faff-a9fee078f367"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://git.io/JtY8I\" width=\"500\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 67,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='https://git.io/JtY8I', width=500) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 413
    },
    "id": "tPDHh1jQMq2n",
    "outputId": "1e4655f9-4e9e-440b-cb25-cce30fbc1b33"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://git.io/JtY8L\" width=\"500\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 68,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='https://git.io/JtY8L', width=500) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rd7Krgw1Mq2n"
   },
   "source": [
    "## L1 규제를 사용한 희소성"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 416
    },
    "id": "wOVmTrIAMq2n",
    "outputId": "4cd2035e-5a1b-4bc8-ebd9-af9f46fbec4b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://git.io/JtY8t\" width=\"500\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 69,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='https://git.io/JtY8t', width=500) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vGpU-439Mq2n"
   },
   "source": [
    "사이킷런에서 L1 규제를 지원하는 모델은 `penalty` 매개변수를 `l1`으로 지정하여 희소한 모델을 만들 수 있습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nTK-tXGvMq2n",
    "outputId": "1543ca34-537c-4e5f-b6c2-d49a3f1127f8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(penalty='l1', solver='liblinear')"
      ]
     },
     "execution_count": 70,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "LogisticRegression(penalty='l1', solver='liblinear')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "di1OmT2VMq2n"
   },
   "source": [
    "표준화 전처리된 Wine 데이터에 적용해 보죠."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "X3NEjPaQMq2o",
    "outputId": "a11d9bb3-7de9-4d8a-9aed-f7a2df004db6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "훈련 정확도: 1.0\n",
      "테스트 정확도: 1.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "lr = LogisticRegression(penalty='l1', C=1.0, solver='liblinear', random_state=1)\n",
    "# C=1.0이 기본입니다.\n",
    "# 규제 효과를 높이거나 낮추려면 C 값을 증가시키거나 감소시킵니다.\n",
    "lr.fit(X_train_std, y_train)\n",
    "print('훈련 정확도:', lr.score(X_train_std, y_train))\n",
    "print('테스트 정확도:', lr.score(X_test_std, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZJjtrnEMMq2o",
    "outputId": "e2ab8ff2-53fd-400d-9abb-1a2f850d43aa"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-1.26392152, -1.21596534, -2.37040177])"
      ]
     },
     "execution_count": 72,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "id": "hmnO2EWTMq2o"
   },
   "outputs": [],
   "source": [
    "np.set_printoptions(8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5S5DwcSxMq2o",
    "outputId": "3b34ff7c-0c2f-4210-b4c7-ecdb83c54ffb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(23,)"
      ]
     },
     "execution_count": 74,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr.coef_[lr.coef_!=0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Y-c8Tjt9Mq2o",
    "outputId": "afc43b15-8118-404f-dfac-4d9d0624d1f5",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.24567209,  0.18072301,  0.74682115, -1.16438451,  0.        ,\n",
       "         0.        ,  1.1595535 ,  0.        ,  0.        ,  0.        ,\n",
       "         0.        ,  0.55864751,  2.50891241],\n",
       "       [-1.53644846, -0.38769843, -0.99485417,  0.36489012, -0.05989298,\n",
       "         0.        ,  0.66853184,  0.        ,  0.        , -1.93460212,\n",
       "         1.23246414,  0.        , -2.23212696],\n",
       "       [ 0.1355558 ,  0.16880291,  0.35718019,  0.        ,  0.        ,\n",
       "         0.        , -2.43768478,  0.        ,  0.        ,  1.5635432 ,\n",
       "        -0.81834553, -0.4930494 ,  0.        ]])"
      ]
     },
     "execution_count": 75,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 285
    },
    "id": "dOYWLvYvMq2o",
    "outputId": "50459078-9d46-49e7-8044-221b294a4679"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmUAAAEMCAYAAACFnMEoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeXxU1f2wnztLlslk3zcICSQQAhEStmiAIipVEqWoqCwKWkVFKyjqr7XqT2tbcHtfwCrqSykBRClaKG5IpYBEkEV2SEhCyL7vmSyz3PePm0xmskAQQgKcx8/93Jlzz7333Bgmz3zPOd8jybKMQCAQCAQCgaB3UfV2AwQCgUAgEAgEQsoEAoFAIBAI+gRCygQCgUAgEAj6AELKBAKBQCAQCPoAQsoEAoFAIBAI+gBCygQCgUAgEAj6AJreboBAIBAIrk8OHjzop9FoPgZiEEECwfWBBThuMpkeiYuLK2l/UEiZQCAQCHoFjUbzcUBAwBBfX99KlUolkmYKrnksFotUWloaXVRU9DGQ3P74NSFlPj4+clhYWG83QyAQCK4qDh48WCbLsm8vNiFGCJngekKlUsm+vr7VRUVFMZ0dvyakLCwsjAMHDvR2MwQCgeCqQpKkc73cBJUQMsH1RsvvfKfd9aIPXyAQCATXNSkpKR6SJMX9/PPPTgBpaWkOgwYNGvpLrhUcHDyssLCw2wGPZcuWec+ZM6ffL7mX4NpDSJlAIBAIrms2bNjgNXLkyLo1a9Z49XZbBNc3QsoEAoFAcN1SXV2t2r9/v/7vf/979hdffNFBykwmE48++mjIoEGDhkZGRka/8cYbfgCbN292HTJkSHRkZGT0PffcE9bQ0CC1nrN06VK/6OjoIZGRkdGt0bfi4mL15MmTIyIjI6NjY2MH79u3z/nKPaXgakFImUAgEAiuW9avX+8xceLE6uHDhzd5enqadu/erbM9/vbbb/vm5OQ4nDx58kR6evrJRx55pNxgMEiPPfbYgE8//TQzPT39pMlk4s0337ROmPDx8TGdPHny1Lx580r/+te/+gM8//zzQbGxsYb09PSTr7/+ev6DDz444Eo/q6Dvc00M9BcIBALB1c28eYQeP47uwjW7T0wMhlWryD1fnc8++8zr6aefLgGYPn16RUpKitezzz5rzR/1/fffu82fP79Uq9UC4O/vb/7xxx+dQ0JCmoYPH94E8NBDD5W/9957fkAJwAMPPFAJMHr0aMOWLVs8AX766SfXTZs2ZQAkJyfXPvroo5qKigoRGBHYIaRMIBAIBNclxcXF6r1797qmpaU5L1iwALPZLEmSJC9atKhDUs+LwcnJSQbQaDSyyWSSLlRfIGhFSJlAIBBcQUwmE7m5uTg6OuLl5YWTk1NvN6lPcKGIVk+QkpLiOW3atIr169dbU4OMGjUq6uzZsw6t72+++eaalStX+kydOrVGq9VSXFysjo2NbczPz3c4fvy4Y0xMTNOaNWu8ExMTa893rzFjxtT+/e9/937zzTcLt27d6urp6Wny8vKy9OTzCa4+hJQJBAJBD2M0GsnMzOTkyZOkpaXR3NxsPebs7IynpydeXl4d9nq9HkkSgZaeYuPGjV6LFy8usi278847K//yl78Etr5fuHBhaXp6uuPgwYOHajQa+cEHHyz9/e9/X/rBBx9k33PPPRFms5nY2FjDc889V3q+ey1ZsqRg5syZYZGRkdHOzs6W1atXn+2p5xJcvUiyfPXn7YuPj5dF8liBQNCXMBqNZGRkcPLkSdLT02lubsbZ2ZnBgwcTFRWFxWKhoqKCyspKKisrqaiooLq6GtvPZI1GY5W09sLm7u6OWq2+pDZKknRQluX4S33WX8qRI0eyY2Njy3rr/gJBb3HkyBGf2NjYsPblIlImEAgElwmj0ciZM2esImY0GtHpdMTExBAdHU1YWNh5RcpsNlNdXU1FRUUHYcvMzMRkMlnrSpKEh4eHnbDZCpyDg0OX9xEIBH0TIWUCgUBwCTQ3N1tF7MyZM1YRGz58uFXEVCqVte7WrVv517/+hdFoxMXFBb1ej4uLS4ettXzgwIG4uLig0+ms16irq7MTthMnTtDY2GjXLr1e32W3qLOzs+gWFQj6IELKBAKB4CJpamqyEzGTyYSLiwuxsbFER0fTv39/q4jJskxqaipr167l008/paKiAm9vb9zd3amvr7duFzOURK1Wd5A4Dw8PvLy88PDwQK/XYzAYqKqqIicnp4OAqdVqnJ1F7lKBoK8hpEwgEAi6QVNTE2lpaZw6dYqMjAxMJhN6vZ4RI0YQHR1Nv379rCIGkJGRwdq1a1m7di2ZmZk4Oztz1113MWvWLG655RZa816BIm4NDQ12klZfX09dXV2HsvOVFxQUcObMmQ7lGo2m03FpAoGgbyGkTCAQCLqgsbHRTsTMZjOurq6MHDmSoUOHEhoaaheFKisr47PPPiMlJYW9e/ciSRKTJk3ij3/8I9OmTcPNza3T+0iShE6nQ6fT4evr22mdX4rFYulU+Orr61m3bt1lvZdAILg0hJQJBAKBDY2NjZw+fZpTp06RmZmJ2WzGzc2N+Ph4hg4dSkhIiJ2INTY2snXrVlJSUvjqq68wmUzExMSwZMkSHnjgAUJCQnrxaUClUlm7OAUCQd9GSJlAILjuaWho4PTp05w8eZKsrCwsFgvu7u6MHj2a6OhogoOD7UTMYrGwe/du1q5dy8aNG6muriYwMJBnnnmGWbNmERsb24tPI7gYJEmKS05Orti8efNZUGbQ+vn5xd5www31O3bsyOjqvK1bt7q+/fbb/jt27MhYt26d+4kTJ5z//Oc/F3VV/5eya9cu3apVq7xXr17dIblucHDwsAMHDpwKDAw0dXau4OpDSJlAILguMRgMVhE7e/YsFosFDw8Pxo4dS3R0NEFBQR0GyJ86dYq1a9eybt06zp07h4uLC9OnT2fWrFlMmjTpkvOGCa48zs7OlrS0NOe6ujpJr9fLX3zxhZu/v7/xYq4xc+bMaqC6J9o3fvx4w/jx4w09cW1B36PPSpkkSdlALWAGTL2Z4FAgEFwb1NXVkZaWZhUxWZbx9PRk3LhxREdHExgY2EHEiouL2bBhAykpKRw8eBCVSsWtt97KG2+8wV133SW6Ba8BJk+eXL1x40aPuXPnVn7yySde06dPr0hNTdUD7NixQ7dw4cJ+TU1NKicnJ8vq1avPxsbGNtmev2zZMu8DBw64rFmzJic3N1czb968/jk5OY4AK1asOHfLLbfU29afOXNmvyNHjrg0NjaqkpKSKt99990CgJ07d+qeeeaZfgaDQeXg4CDv2rUrbc+ePS6tEbmioiL19OnTw4uLix3i4uLqroXk7wJ7+qyUtfArWZZFtmeBQPCLMJlM5OTkkJmZSWZmJsXFxQB4eXlx4403Eh0dTUBAQAcRMxgMbN68mZSUFLZt24bZbGbkyJG888473H///QQEBPTG4wh6iNmzZ1e88sorgTNmzKg6deqU7uGHHy5vlbLY2NjG/fv3n9ZqtfzrX/9yff7550O+/fbbzK6uNX/+/H6JiYm1L7/8cqbJZKK6urpD+PSdd97J9/f3N5tMJhISEqL27dvnHBsb2zhz5syIdevWZU6YMMFQUVGh0uv1dmtjvvjii0Hjxo2re+uttwo3bNjg/tlnn/lc/p+GoDfp61ImEAgE3UaWZcrKyqwSlp2djclkQqVS0a9fP26++WYGDRqEn59fBxEzm83897//JSUlhU2bNlFXV0doaCiLFy9m9uzZREdH99JTXR/M2zwv9HjJcd3lvGaMX4xh1Z2rLrjQ+ZgxYxry8vIcP/roI6/JkyfbdUNWVFSoZ8yYMSA7O9tJkiTZaDSeN+tuamqq6z//+c+zoCyT5e3tbW5f5x//+IfX6tWrfUwmk1RaWqo9cuSIkyRJ+Pn5GSdMmGAA6Gyx8r1797p+/vnnGQD33Xdf9WOPPdbh2oKrm74sZTKwTZIkGVgpy/KHvd0ggUDQ92hoaCArK8sqYjU1NQB4e3szcuRIIiIiCAsL63LZoaNHj7J27VrWr19Pfn4+bm5u3HvvvcyePZvx48fb5R4TXLtMmTKl6pVXXgndtm1bWklJifVv4wsvvBA8YcKE2u+++y4zLS3NYdKkSVGXcp/Tp087rFixwv/gwYOnfH19zdOnTw9rbGwUv2QCoG9L2U2yLOdLkuQHfCdJ0mlZlne1HpQk6VHgUYB+/fr1VhsFAsEVxmKxkJ+fT0ZGBpmZmRQUFCDLMo6OjoSHhzN+/HgiIiLw8PDo8hr5+fl88sknpKSkcPToUTQaDb/+9a955513SEpK6tFs9/XA6Za9ud1musD7y13Wl+hORKsnefzxx8s8PDzMo0ePbti6datra3lNTY06JCSkGWDlypUX7C688cYba998803fl19+uaS1+9I2WlZZWal2dna2eHl5mXNzczX//e9/3SdMmFA7fPjwxpKSEu3OnTt1EyZMMFRWVnbovhw7dmzt6tWrvZcuXVr42WefudXU1IiZJdcYfVbKZFnOb9mXSJL0BTAa2GVz/EPgQ4D4+Hgx2lEguIapqqoiIyODrKwssrKyaGpqQpIkgoODrRIWHBzcZVTLbDZz7Ngxdu/ezZYtW/jPf/6DLMuMGTOG5cuXM2PGjMuetFUGCoDDwBGb/ZmWYz2FCuWDXd1u66xM0EZERITxpZdeKmlf/sILLxQ98sgjA5YsWRJ0yy23VF3oOu+//37OQw891D8yMtJHpVKxYsWKc5MnT7YO9B83blxDTEyMISIiIiYwMLA5Li6uDsDJyUlet25d5tNPP92vsbFR5eTkZNm1a1e67bX/+te/FkyfPj184MCBQ+Pj4+sCAwObL8ezC/oOUl+cvSFJkgugkmW5tuX1d8Brsix/01n9+Ph4+cCBA1e0jQKBoOdobm4mOzvbKmLl5eUAuLm5ERERwcCBAxkwYECXEa3Gxkb279/P7t272b17N6mpqdZuzfDwcGbOnMmsWbOIjIy8PO0FTqFIl62AldvUGQDEtmzDAA8uLE7dlavWMhVwMcuMS5J0sDdnth85ciQ7NjZWTOYSXHccOXLEJzY2Nqx9eV+NlPkDX7QMxNUA67sSMoFAcPUjyzJFRUXWcWE5OTlYLBa0Wi1hYWHEx8czcOBAvL29OwzQByWSlpqaapWw/fv309ysBBGio6O5//77SUxM5KabbqJ///6X1NZyOsrXSaA1sZUTEANMo03ChgPul3RXgUBwPdAnpUyW5SyUzzKBQHCNUldXZ5WwrKws6uuVHh5/f3/Gjh1LREQE/fr1Q6Pp+DGVn5/P7t27+eGHH9i9ezfHjh1DlmU0Gg1xcXE8/fTTJCYmkpCQgI/PL8saYAEysJevI0CeTZ0AlA+qKS37G4BB9NEPVoFA0OcRnx0CgeCK0FXOMJ1OR0REhHXT6/V258myTFpamjUK9sMPP3D27FkAXFxcSEhIYPr06SQmJjJmzBh0uovPqlAHHMNewI6hDMYHpWtwCDCBtuhXLEpIXyAQCC4XQsoEAkGPYTAYSE9PJy0tjczMTIxGo13OsIiIiA7JW41GI4cPH7aTsLIyZdiRr68viYmJPPXUUyQmJnLDDTd0GknrChkl0tU++pVB2+B7d5SI18Mt+1ggGqVbUiAQCHoSIWUCgeCyUl5eTlpaGmlpaeTm5iLLMq6ursTGxjJo0KAOOcPq6+vZt2+fVcL27t1r7coMDw/njjvusI4Hi4yM7HRMWVeYUOTrB2BPy952xegIFOmaRZuA9ePiBssLBALB5UJImUAguCRa84adPn2a9PR0a1QrICCAxMREBg8ebBcNKysr44cffrCOBzt06BAmkwlJkhg+fDhz5861SlhQUNBFtaUW2EubhO2lrQsyDLgZGIciYMMAt0t+eoFAILh8CCkTCAQXjdFoJDMzk7S0NNLT0zEYDKhUKsLCwhg1ahSRkZHW5K0NDQ189dVXbN26lZ07d3Lq1CkAHB0dGT16NIsXL7YOynd3v7g5ivm0RcB+QOmKtKCkhogF5gE3tmwhl+fRBdcgKSkpHnPmzIk4dOjQiREjRjR2VU+n040wGAw/X8m2Ca4vhJQJBIJuUVdXZx0flpWVhclkwtHRkUGDBhEVFcXAgQNxclJGXpWUlPD3v/+dLVu2sG3bNgwGA3q9nvHjxzN79mwSExOJj4+31u8OFpRcYD/YbNktx3TAWOAlFAEbi4iCCbrPhg0bvEaOHFm3Zs0arxEjRhT0dnsE1y9CygQCQae0Lu7dOj4sL09JBuHu7s7IkSOJioqif//+qNVqZFnm9OnTbNmyhS1btvDjjz8iyzKhoaHMnTuX5ORkJkyYgKOjY7fv3wgcoE3AUoHKlmP+wE3A71Ak7AZAe9meXHA9UV1drdq/f79++/btacnJyYPefffdgnPnzmmnT58eXldXpzabzdLy5cvPTZkypQ7gqaeeCt62bZu7k5OTZevWrRmhoaGm3n4GwbWDkDKBQGDFYrGQm5trHR9WUVEBQFBQEBMnTmTw4MH4+fkhSRImk4kffvjBKmIZGRkAjBw5kldffZXk5GRiY2O7PTC/HEW8WiXsAEqmfIDBwHQUEbsRZYD+dTMYX5bBIoNsadnLNnvLRRzv5L2A9evXe0ycOLF6+PDhTZ6enqbdu3frtm/f7nrzzTdXL1mypMhkMlFbW6sCaGhoUI0bN65u+fLl+fPnzw9Zvny579KlSwt7+xkE1w5CygSC65zm5mYyMjJIT08nPT2dhoYG1Go1AwYMYNy4cURGRuLmpnQG1tTU8M9//pMtW7bw5ZdfUllZiYODA5MmTWLRokUkJSUREnLh0VsykIX9eLBTLce0QDxKFOwmIAH4ZelfexGLBYwmaDZCswmMxpbXNu8t7SWqi/c9xUXMYr0izJsXyvHjF59k7nzExBhYdf6Fzj/77DOvp59+ugRg+vTpFSkpKV7Tpk2reuyxx8KMRqPq7rvvrkxISGgA0Gq18n333VcNEBcXV799+3bRSy64rAgpEwiuQ2pra63dkmfPnsVsNuPs7GwdHxYREWHtaszJySElJYUtW7awY8cOjEYj3t7eJCUlkZyczK233oqrq+t579eamsJWwlpTU3igiNdsFAmLBzpf0bKXkWVFqqyyZexaukxd9GipVaDVgoMGVCrQqEAlKYKkUrXsW963L1NJINnWb3lve45tnU7rt1wT+p6U9QLFxcXqvXv3uqalpTkvWLAAs9ksSZIkf/DBB3m7du1K27Rpk/u8efMGLFiwoHjBggXlGo1Gbl30XqPRYDKZxA9RcFkRUiYQXAfIskxJSYlVxAoKlLHMnp6ejBo1iqioKPr164dKpUKWZQ4dOmTtljx8+DAAgwYN4ne/+x3JycmMGzfuvElbq7FPTbEPMLQcCwMmo3RD3oSSmFXVI0/dDWTZXrI6CJdNmbEL0VKpwKFFtHRO4N7y2kGrbFpt23G1+so+39XEBSJaPUFKSorntGnTKtavX3+utWzUqFFRX3/9tf62226re/bZZ8uampqkQ4cO6bBfX14g6BGElAkE1yCyLFNeXk5ubi65ubmcPXuWqqoqAEJCQpg0aRKDBw/Gx8cHSZJoampi27ZtVhHLz89HpVKRkJDA0qVLSU5OJioqqvN7ATm0CdgelCWKZBTZas2O3+OpKWQZzC3dhiaTzd5sX9ZewjpDJbUJlZMjuLl0FKxW6RKiddWyceNGr8WLF9vmE+bOO++sfPTRRwfodDqLRqORdTqded26dWd7q42C6wtJvgYGe8bHx8sHDhzo7WYIBL2G0WikoKDAKmG5ubk0NDQA4OTkRL9+/YiKiiIyMtK6tmRZWRlfffUVW7Zs4dtvv6Wurg4XFxduu+02kpOTuf322/H19e1wLxNKPrA9Nlt+yzFXlHQUrQI2pqXsomkdk2XqRKi6KjeZzz94Xa0GrRq0mnaC1SJZtu/Vquuie0+SpIOyLMf31v2PHDmSHRsbW9Zb9xcIeosjR474xMbGhrUvF5EygeAqpKamxk7AioqKsFgsAPj4+BAVFUVoaCihoaHWaBhAenq6NRq2Z88eLBYLQUFBzJo1i+TkZH71q191yB1Wg9IV2SpgtlnyQ4FE2mZFDkNZvLsDFgsYGqHJ2E6k2klWa5nZ0vXDS5IiVloNaNRKl6FWAxqNfbltmUbdNpZKIBAI+ihCygSCPo7FYqGoqIjc3Fzy8vLIzc2luroaUAYbBwcHk5CQQGhoKCEhIeh0bRPY6urq2L59O9999x1btmwhLS0NgNjYWP7whz9w5513MnLkSLu0FTm0CdgPKF2RrVnyhwNzaYuEhXbWYFmG+gaoNUBdPdTWQ11D51EsW3ly0IKLc1uZnVRp2qJcqusjiiUQCK4/hJQJBH2MhoYGq3zl5uaSn5+P0WgEwNXVlX79+jF27FhCQ0MJCAhAbTOmqby8nO+++45du3ZZ15U0m81otVomTpzIggULSEpKon///kDHBbv3AHkt19KjdEX+kbauyA7z/2VZiYDVGRT5ahWwlqgdajW46iDEH/Q6cHSwj2YJuRIIBAIrQsoEgl6k/YD8vLw8SktLAZAkiYCAAEaMGGHtimy/NmRubi67d+9m9+7d7Nq1i5MnTwJt60q++OKL1nUlXV1drQt2/522rsi6lmuF0BYBuxElKmb3ASHL0NikRMCsAmZo62pUqRQBC/RV9q4u4OwoxEsgEAi6iZAygeAK0jogPycnxxoNsx2QHxoayrBhwwgNDSUoKAgHBwfrubIsk5aWZo2C7d69m+zsbECJoN14443MnDmT8ePHEx8fj6OTE5nAT8DvUSSsdcFuCUW65tCWmqKfbUNlGZqaOwqYyawclyQl8uXv0yZgOichYAKBQHAJCCkTCHqQXzogH8BsNnPo0CFrFOyHH36gpKQEAD8/PxITE3nmmWcYP348w4cPp0Kt5ifgO+BPwH6gouVaLijdj39AEbAOC3bbClhdvfK6NV2EJCljvXw9FflqFTAxcP6qpanBSFVx/YUrCgSCK4qQMoHgMiHLMpWVlZw7d46cnBzOnTtHZaWyhPaFBuQDNDY2sn//fmsUbM+ePdTW1gIQFhbGlClTSExMJDExkZDISH6WJH4ClqAkZ81uuY4KiAF+A4xu2YZi84+92WgzCL9FxJqNbQ1xcQZvd9C3CJjeWQjYVYDJaKaqpJ6q4noqi+qoLKqnqqiOyuKWfctWVVxPfXVTbze3T5GSkuIxZ86ciEOHDp0YMWJEI0BaWprD1KlTB505c+bExV5v+vTpYVOnTq2eO3du5YwZM/o///zzxXFxcY1d1V+6dKmvTqezLFiwoHzZsmXeycnJNWFhYcau6neXb775Rr9gwYL+Go1GPnDgwCm9Xv+LcmCNHj066q233sodP3684cK1BZeCkDKB4BciyzKlpaV2EtYqUc7OzvTv359Ro0bRv39//P397Qbkg7LUUWpqqrU78qeffqKpSfljOXToUGbNmqWMB0tMpDYkhH0oXZHvocyIbOlIpB9KFOxJFAEbiTJIH1Bkq87QMhC/RcCamtsaoXMCD9eWCJhO6ZIUyVD7DBaLTG25wSpZrVJVaZWseqqKlX1NWed/L3VujngGuOAZoGdArD+eAXo8A/R4+Lvw5SMvX+En6pts2LDBa+TIkXVr1qzxGjFiRMHlvPann3567kJ1nn/++dLW12vXrvW54YYbGi6HlK1Zs8Zr0aJFhU888UTFhWsL+gJCygSCbmKxWCguLiY7O9sqYa3jwVxdXenfv791a98VCVBaWmo3KP/w4cNYLBbUajVxcXEsWLCAxPHjGXDjjaR7e/MTsBL4LW15wdxRxOt/WvajgABoGwPWKmCtW5PN53prZnpXvxYBc1FmQAp6FFmWsVhkLCYL5pbNZLRQU2boIFlVRfV2ka2qknos5o7BDUdnDR4tchU0yIvom/pZxcvD38VOvBydtV037pEefPCrhOrqatX+/fv127dvT0tOTh707rvvdpAyk8nEE088EbJjxw53SZLkBx98sOwPf/hDyXPPPRf4zTffeDQ1Nani4+Pr1q1bd07VLqpsG2XS6XQjHn744ZJt27a5Ozk5WbZu3ZoRGhpqWrRoUZBerzcPGDCg+fjx47o5c+aEOzk5WV599dX8jz/+2Gf79u2ZAF988YXb3/72N9/vvvsu0/Yemzdvdn3xxRdDzWYzsbGxhjVr1px7//33vb/88kuvnTt3un/zzTfuW7ZssVuVYPLkyRGFhYUOTU1Nqvnz5xc/99xzZSaTiRkzZoQdPXrURZIkeebMmWWvvPJKCcAnn3zi+eSTT/avra1Vf/DBB9lTpkypQ3DZEVImEHSB2WymoKCAc+fOWaNhzc1KlMnT05OoqCirhHl4eFglzGKxUFBQQFZWFhkZGfz444/s3r2b06dPA8qA/nHjxvHSSy8xIjER9dixHNXr+QlYR9tC3Q4oSxTNRYmEjQYGAirbNBS2W+sgfGhZg9FViXy1btrr65+7LMvUVjRQlldDWW4NZXm1lOfVUFPegNlkwWJWBKlVlixm2SpNtu/bjnc8Zj1utpeu9nW7g0arwsNfj2eAC17BrkSMDFDEKkApaz3mGaDHWe/QQfoFv4z169d7TJw4sXr48OFNnp6ept27d+sSExPtwo5vv/22b05OjsPJkydPaLVaiouL1QCLFy8ueeuttwoB7rrrrgEbNmxwf+CBB6q7uldDQ4Nq3LhxdcuXL8+fP39+yPLly32XLl1a2Hp87ty5le+//75fq8RZLBb+53/+J6SgoEATFBRkWrVqlffcuXPtVkAwGAzSY489NmDbtm1pw4cPb5o2bVrYm2++6fvyyy+X7NmzR9/ajdq+LevWrcv29/c319XVSSNGjIieNWtW5ZkzZxwLCwu1rV22ZWVl1m9tJpNJOnbs2KlPP/3U/bXXXguaMmVK+i/9mQu65vr6lBYIzoPRaCQvL88qYXl5eZhMymB3X19fhg0bZpUwSZLIysri7NmzfP/992RlZVm37OxsazckgIeHBzfddBOz587FPzGRurg4Djo48Cnwms39o4BbUORrDMrsSEeLRcn7ZStf9TZ5wCRJGfPl69kmXy7O13wXpCzL1JQZFHhigSwAACAASURBVOHKq22RrpoOAtbcaL+2pUotofd0RqNVodaoUGla9moJdctr5X3bca2TBscOxyW7823P6apO+3u4+ejsolp6T6frWrTmQehx0F24ZveJAcMqOO9C55999pnX008/XQIwffr0ipSUFK/2Uvb999+7zZ8/v1SrVaKO/v7+ZoCvv/7a9Z133globGxUVVVVaaKjoxuALqVMq9XK9913XzVAXFxc/fbt2zuk/rNFpVJx7733ln/00UdeTz75ZPmhQ4f0n3/+uV3E68iRI04hISFNw4cPbwJ46KGHyt977z0/oOR8116yZIn/l19+6QFQVFSkPXHihNPw4cMbc3NzHR988MHQpKSk6mnTptW01r/nnnsqARISEuoXL17s0NV1BZdGn5UySZKmAP8XZdWWj2VZ/msvN0lwjdHY2Ehubq5VwgoKCrBYLEiShL+/P4MGDQKwDt7ftWsXZ8+eJSsri7Iy++X63N3dCY+IYGBMDDcmJ+MZHo5reDjaAQMojIjgJ5WK/wVaR3P5o4jXbNq6IT1Mps4FrBW1WhGwQJ82AbsGZ0FaLDLVpfWU59W2E6022SrLq8HYZLY7T61R4RXkim+oGwPjAhh7ZyQ+IW74hLrhE+KGd4grnv561Jpr6+cl+OUUFxer9+7d65qWlua8YMECzGazJEmSbLFY8i50rsFgkJ599tn++/btOzlw4EDjokWLghobG8/7y6XRaOTW7k2NRoPJZLqghT/++OPld9xxx0AnJyc5KSmpslUML4WtW7e67ty50/XAgQOnXV1dLaNHj45qaGhQ+fr6mo8fP37yiy++cPvggw98P/30U6+NGzdmAzg5Ocmt7Tabzdfvt4cepk9KmSRJapTxzLegJBjfL0nSFlmWT/ZuywRXM/X19daxYDk5ORQVFSG3LP0jSRI1NTXk5ORw9OhRMjMzrVEyALVGg2f//riGh+M3fTpB4eEwYADN4eHUh4dT5enJz8DPndzXBYgHfkfbbMjQpmYkq3y1iFijzYw4B60iXd4ebQLm5HDV5wGzWGSqS+opbZGs8nbRrbK8GsrzazE12wuXRqvCO1gRq0Gjghj3m8GKcIW44h3ihm+oG+5+LqjVQriuVi4U0eoJUlJSPKdNm1axfv1662D8UaNGRX377bf68PBw64yYm2++uWblypU+U6dOrWntvmyduBMQEGCqrq5W/fvf//ZMSkrq0E14sej1enN1dbU11B0WFmb09/c3vv3224HffPNNhy7D2NjYxvz8fIfjx487xsTENK1Zs8Y7MTGx9nz3qKqqUru7u5tdXV0tP//8s9ORI0dcAAoLCzWOjo6Whx56qGro0KGNs2fPDr/U5xFcHH1SylD+bmXIspwFIEnSBuBOQEjZVYR1gLNZbhu/08lr69idTl5bzDImo7ndMRmzyWz3vu2abe9ra+s5m59LaXkpjYY6VLIFVCDLZsqrKikuKqSktJiy8lKMZiOSuxtycADmfjfiMu4e5NAg5H4hyP1CITAAg5OWBrUKN7WEh0aFh1pFiFrCU1bhVdOEl1rCS63CR6PCWy3hI0l4yTKhjU1oWmc/toqY0aZbzclRkS7bCJjDpX8bbo/ZbMHYZMbUZMLYZMbYYd9Zmf0x0yXUb240UVVcj9lkv9i4xkGNd7AS4Ro8LgSfEFd8Qt2VfYgS5XL3c0GlurqFVND32Lhxo9fixYuLbMvuvPPOyrVr13q9/PLL1vKFCxeWpqenOw4ePHioRqORH3zwwdLf//73pTNnziwdMmTIUF9fX1NsbOxlSfw2Z86csqeeeqr/4sWLLa1pLO67777y9957TzNy5MgOaTV0Op38wQcfZN9zzz0RrQP9n3vuudLOrt3K9OnTqz/88EPf8PDwoeHh4Y2tbc/OztY+/PDDYRaLRQJ47bXXLhgxFFxeJLmzRYJ7GUmS7gamyLL8SMv72cAYWZYXdFY/KChILiws7OxQB37729/y4Ycf2pU9+uijfPTRR906/5VXXuHVV1+1K0tKSmLr1q3dOv9vD0k8MtH+2/yYV8z8fMFJ0wqf/07F1BH2f5z6P2OmsKp75+99VcXIMPvzHR4yd1G7I9nvqgjybDu/oFImbKHlPGfY8/6tt1NT40J1rY7qGheyKmv5ui6lW+c64spkFtuVFXOaA6zv1vluBJLI43ZlORzgGFu6db4fkYxill1ZOt9zhv926/x+qjiGa+60Kztq2kyO5WC3zh+kmkiUZpJd2U/GtZTI3RtvO4xk+hFvV7ab96mhe/924nkAfwYDIDmAygG21b9Jo3zeL+VWpk15gv6DA3ALacIruB7v0Fpm3/T/unUuQH6+lqAgm9+9Apng4O5nDZDl1mEwakDHwYNa4uOLzneKlcBAdwoKlqHEPXWAC//+90GSkxd16/yRI0dy8KD9/+cPP/yQxx57rFvnT506lX//+992Za+++ir/+7//263zz/O5d1CW5fguTutxjhw5kh0bG1t24ZrXN3PmzOk3YsQIw8KFC8XP6hrhyJEjPrGxsWHty/tqpOyCSJL0KPAo0GE9wL5MWnEkO84E25XVNu0HuveH7VhBDC56H7uyJtMPtI1WOj8HckZSbWw/tvT7bp0LkHp2HN5ljtb35bVNKAv4dI/5D35l9/7gWfj6pe6d66Q3M2lRDQaLDoukRVJDWrrEge45GT7haub9VYfFAmYzmGXY/h8Hjv29e+f7RWm4faYrsgVkM0oUb6cDZ7r5+H7hEmPHoKxx1ELRXsjJ6N75IcNg7EibAgnOfAcl3ez0GTgREkZIqDWg0YJaA8c+lqjpZlamO/8gkXijhFqLdUD6nhnQWN698++4U6Z/nAaDuwP1Og8MjhfX1XigYSFhTS7oJAsuKgs15mpg2UVcoVWgTChJRnKAL7t5bjXw4EXcqz1ngLuwlTo4dRHn5wMbUT6ytS37rIs4vxHlGVrPvfyRWEHPMHTo0CHOzs6WlStXXvHuXcGVp69KWT4QavM+pKXMiizLHwIfghIpq67ucsJLnyL6rkXc+uijdmVuX8dB0aFunT9i9p+4NSnJrsxxVRDUdS/aMfaJD4iLi7MvfKP73UITXviMoKAg6/uCggJYFnyeM+xZ7XIGZ8difNyKcVGVk9lwFFjRrXN1GgOLot5S3mj04OjLv9Gwrpv3djXCr779GfXho+hOpqNpaKIOZXHu7nBcPsmTloXKG7WySdru/+yGTRjGHz7+o13ZuUdzOZDRvUjZ+Lsm8IdX7c/fm/QTabndi5RNuf8OHm33u/f/tn5BTkH3rCx2xAwmjEuCJpS/8U3goHqP7n6huOHAw8TtilMGIaQp13iEZ7t1LsDhJhfOGtq+gFXXXtzHl1n+E2rJdlbqQbovZf4oXz7qbbYdwJ+7eb4FONvu/IbznmHPz8C9F1G/PSktm+Bq48SJExdj74KrnL7afakB0oGbUWRsP/CALMudLncRHx8vHzhw4Aq2UHCpyLJMtcHMmRwTeaUmqhpMWLQmnD1NOOnk1koYq2qoyq5GrqrAxVSJt7YSP10FAe5leLuU4SSVIjWVQmMpNJWCpV3E0D0GBsyC/g+Ai43nm82QlgaHDinbwYPw88/QkpEfBwcYNgx55EiaY2Ooig6nJNyfcrmeyoZKKhoqqGxs2TdUUtFY0fa65VhVY8c+5WDXYAZ6DeywRXhG4Oro2kM/7T6IGWVdqJPttlO0ZcoF8AOilc0y1EJDTAP1UfUYPA3Uy/UYZAMGi7LVy/XWfbPceeTYWXJGr9LjqfLEW+2Np1rZu6vc2wnblURG+YGYAGPL3vZ1T5UZkaQ3RPelQNALdNV92SelDECSpNuB/4MSk1gly/IbXdUVUnbtIMsyjWYLBWUmsgtNlNaYaMKERm/CWd82dq3RIJGXpaEkV0NzrQYnNPi4qhkYaGBIeBkDAktxMhyE7HVQ9iMggd8ERdBCp4ODR8ebWyyQmakIWqusHToELetXotHA0KEwcqSyxcVBbCzoOk+tZLaYqWys5FzVOTIqMpStMsP6uqjOfjyTv4t/p8I20GsgHk6dtPdaxIIy37q9rJ3EPvuTJ1ZZs25DUGLqEphkk52oGWQD9RbldY2lhgpLBbWWtgifChXuKne81F7KplL2nmpPtNK129UnSZKQMoGgF7jqpOxiEFJ2fdBkslDdaOJckYnCSiN1zSZwNOHk2iZrxmYoOKshL1NDdYkGS50DA/Q53BKxgWGua3ExpyOrHJGCkyBsJgT9GtSOXd9UliE7uy2a1rpvzVOmUsHgwYqgtcraiBHgeuGoV21TLZmVmW3CZrPl19r11uPt7N2lsHk7e1/7SUdloBAlkmYraicA2zFtrihy1l7Y+qOs1G6DUTZSYa6g0lxJhaWCCrOyVVmqkGn7XHRTuVklzVbaHFXn+b25ShBSJhD0DkLKBNcsRrOF2mYTZbUmcktMVBpMmNQmHPTKrFKLBfIzNaQf0UD2cUbrPyMp+lO8XEoxmD3JV92Lud8sgoYn4ObejcHnsgx5efbRtIMHoXUGsCTBoEFt0bQ77oAhQy7qmQxGA1mVWZ0KW051jp00eDh5tEmap72w+bn4XfvCVkpb16etsNkOs3QGBqMIWhTKiFXbzbmtqlk2U2WpskpahbmCCosib2baZiq7SC7W7k9PladV2HSS7qr5mQspEwh6ByFlguuOZrOFykYjFQ3NFFYbqW5uRlYpv+/GBhOq06kElW5ipNuXOGsNnC0JY/PRmfxcORNn/yFERWHdwsK6sXJRYaEyLs22+zMnRzkWHw+zZ8N994Gf3yU9V5OpibNVZzsVtuyqbMxymzjoHfQM9BpIpHckMb4xDPcfzjD/YYR5hKGSrvFEq5UootZe1nI6qeuNImf96ChsoUAwWDQWpeuzRdJaha3SXEmzzexnR8mx08iaq8r1kmRNlmXMmDHJJowYMcpG5bVstH+PTXnLMdt6ra9NsonZHrOveymTJCkuOTm5YvPmzWdBWW7Nz88v9oYbbqjfsWNHN+dGX36ys7O18+fPD/3mm28uZpqt4CpBSJngukeWZeqNZsobmqlokbWaJhMqcz1BVd8QUvI5/oadqLBwsmQE/2/7LNbvuZ+iqkAcHGDgQOxErXXz8jrPTQsL4dNPYc0aRdg0GpgyBebMgaQkcHK6rM9oNBs5V32ug6ydKjtFVmXbZ7uL1oUYvxiG+Q1jmP8wRdb8huGt876s7emTNKGMW8vtZMtp2befoyEBAbRJmo28yaEy9aH1lHuXUylXUm4up9JSSYW5gga5bYalBo1V0jxVngCYsJEn2Wh9bytetu9tI6TdQY0ajaRBixatpFVeS1q0KK+TXJOueynT6XQj+vXr19SaqPWzzz5ze/nll0MCAwObe1PKBNc2v1jKJEkaIMvy2QuV9SZCygS/FJPFQlWjkYoGIxWNzdRV5+Nb9gX9KjbhaTiKjIoih4kcqHyAf+65m3173cnMBJsVmPDx6VzWwsOVSZxWjh+HlBRYtw7y88HdHe65RxG0G2/s8TUs65rrOFFygmMlxzhafJRjJcc4VnyM8oa2QVmB+kCG+Q9TZM1PkbUhvkNw0lxeeezz1NG5tNmKm6HdORogGLsIW8OgBioGVlARXEGFVwUVDkp0rU6uA0BCsgqSVtLaCZNW0qLBprzd+1bZshWt9scuFA0V3ZeKlM2bN68kLi7OMHfu3Mpp06aFRUdHN6ampup37NiRsWPHDt3ChQv7NTU1qZycnCyrV68+Gxsb21RbW6uaMWNGWFpamnN4eHhjcXGxdsWKFTnjx4836HS6EQ8//HDJtm3b3J2cnCxbt27NCA0NNRUUFGjmzp3bPz8/3wHgnXfeybn11lvrv/zyS/2zzz7bD5QcgKmpqadLSko0U6dOHXTmzJkTy5Yt8z5w4IDLmjVrcgB+9atfDXz22WeLp06dWqvT6UbMnj279D//+Y+7n5+f8Y033sh74YUXQgsKChyWLFmSM3PmzKsjX9R1xqUkj90EjGxX9k8grpO6AsFVhUalwkfniI9OGbQtB3nSYIqmomExJeUncMz9BN+SjSS5PMKvpzxFyf2/pjpwBpUOt1KaqyPjmJbTpyTS0mDrVli1qu3aarUiZnFxinMlJMQw/I0laP78Z9ixQxG0Tz6Bjz9W+kdnz1a2loXQLzd6Bz1jQsYwJmSMtUyWZYrqijqI2oqfVtBkVtbiVEtqBnkPshO1a74LVI8yYaCroYAySvdoV+K2F/gnOBudCW75D1DGroWAKcIEbqB2UiPpJeV+epScsvoLbC6AA3ZJiAWXxuzZsyteeeWVwBkzZlSdOnVK9/DDD5enpqbqQVlbcv/+/ae1Wi3/+te/XJ9//vmQb7/9NvPNN9/09fDwMGdmZp7Yv3+/07hx44a2Xq+hoUE1bty4uuXLl+fPnz8/ZPny5b5Lly4tfOyxx0IXLVpUfNttt9WdOXPG4bbbbhuUlZV14u233w5YtmzZuVtvvbW+urpapdPpLCUlJd1qe0NDg6plbc68W265JeKll14K3r17d/qhQ4ec5s6dO0BI2dVFl1ImSdJgYCjgLknSb2wOuQHX2ddmwfWCJEnotGp0Wmdwi4cB8ZjNb1JTuAv57Fr8CjcRVP45jRpv8jzvxH3idEb+ZhRezg54OTugMWo5l6EmPU0RtVOnYNcu2LBBub6LC4wZo+bGGyeTcP9kxv75b3js+ELp3nzjDXj9dRg7VpGzGTPAu2e7EyVJItA1kEDXQG6NuNVabrKYyKjIUESt+BjHSo5xsPAgG09utNa5rrtAJcCrZYvtoo4FKKFTadPkaZQu1DqbrcOqhudBw4XF7UJyp7+I+10J5hHKcTrPL/NLicHAqgsvdD5mzJiGvLw8x48++shr8uTJdhJTUVGhnjFjxoDs7GwnSZJko9EoAaSmpup/97vflQCMGjWqMTIy0ho71Wq18n333VcNEBcXV799+3Y3gD179ridOXPGOq2krq5OXV1drRo7dmzdc889F3rvvfdW3H///ZURERHdXrtOq9XKd999dw3A0KFDGxwdHS2Ojo7y6NGjG1ojcoKrh/NFyqKAqYAHYJtCvhb4bU82SiDoS6jVKtxCJkLIRDD/DQq/RpO1loiCdQwsXYXBKZxzXtM56TWNescwHL1VDLpZy5g7HPBy1uLl5EBerkRqKuzZA6mpin9ZLCBJLgwdOosbb5zFzbfn86vC9Xh/tQbpySfhmWeUmZtz5sDtt4PjlUvBoFFpGOwzmME+g7l3aFsm+dqmWk6UnrCK2rGSY3xx+gs+/vljax3bLtBWUbsuu0BVKOPQAoBR3ajfuvpTq6TZvu5qa1+noJM63V+a9rpmypQpVa+88krotm3b0kpKSqx/G1944YXgCRMm1H733XeZaWlpDpMmTYq60LU0Go2sahmOoNFoMJlMEiiR6UOHDp3S6XR244b+/Oc/F911113Vmzdvdk9MTBz85ZdfntHpdBbb61ksbf8jm5qaVLbHWu+lUqlwdHSUAdRqNWazWcRTrzK6lDJZljcDmyVJGifL8o9XsE0CQd9F7QAhd6IJuROaqyF3E7rstQwueIshBW/S6DGaYt+7yTLfwYk6ZTC3j7MDY4I9uP9+Nfffr1ymrg727cMqap98AitrgoHF+Pk+xwMTjzDTksLwXetw+Ne/wNNTmbk5e7YSSeullAuujq6MDRnL2JCx1jJZlimsK7QTta66QGP8YhjqO1TZ/IYyyGsQWvW1m5z1otAA7i3b5UJGmdjQldRNu4z3ulS6EdHqSR5//PEyDw8P8+jRoxu2bt1qTTRYU1OjDgkJaQZYuXKldeHhcePG1W3YsMEzKSmp9uDBg07p6enOnV3XlptuuqnmL3/5i9/rr79eDJCamuqckJDQcOLECcfRo0c3jB49uuHgwYO648ePO40ePdoaeYuIiGj+6KOPdGazmbNnz2qPHj3qcnmfXtBX6M6YsgxJkn4PhNnWl2V5Xk81SiC4KnBwh4h5EDEPqT4Xzn2CU/Za+p95nv7S7zEH3ka53738aJjAjnNmxgV74u6kCIheDzffrGygrPp08mSrpEn8O/UG/k/mDahZwq8123lKvYaJH63G4f33MQ0YiOah2TBrljJorZeRJIkg1yCCXIO4beBt1nKTxcSZ8jNWSTtWcoyfC39m08lN1lmEWpWWSO9IhvoNtcpajF8MEV4RaFR9dWneqwgJZbCJE+BzgbrXOREREcaXXnqpw0CuF154oeiRRx4ZsGTJkqBbbrnFOi938eLFpffee29YRETE0IiIiMaBAwc2enp6mtufb8uHH36Y+8gjj/SLjIyMNpvN0pgxY2oTEhJyli5d6peamuomSZIcFRXVcPfdd1fn5ORYv63ccsstde+9917TwIEDhw4cOLAxOjq6/TQTwTVCd2ZfpgK7UVbvtf7CybK8qWeb1n3E7EtBn6LyqLK8U/Y6aMjH7BrF/pA/U+ySQHygO8GuF/xCDUBREfz4Y1s0Lf1ADUnGTcwmhUnsAKBw4E1YZs4h8Ol7UHldHUsxGYwGTped5kTJCU6UtmwlJzhb1Tah20HtwGCfwXZRtaG+Qwn3DEet6q01Kq89xOzLX47JZKK5uVnS6XTyiRMnHG+99dbIzMzM405OTld/nilBj3MpKTEOy7J8Q0817HIgpEzQJ7GYoeBLOPgM1J+lyG8GBwJeIjygP0O89RedSLSxUclLm5oKad/lEJa6jun1axjCaRpx5KeAZAomzcb/wSmMStCi72sDuS9AfXM9p8pOdZC1c9XnrHWcNE6dytoAzwHX7kzQHkRI2S+nsrJSlZiYGGU0GiVZlvnTn/6Ud++999b0drsEVweXImV/AlJlWf6qh9p2yQgpE/RpTAY48QbyqTcxq/QcCf4Dxv4PEh/kheYScpPJMmRmyJxedxDnTSmMOLUeL3MZJfjyqXQf+yLn4Dk5joQbJcaPh+Dgy/hMV5DaplpOlZ3ieMlxO2HLq8mz1nHWOBPtG23XDTrUbyj93PsJWTsPQsoEgt7hUqSsFmWCdXPLJgGyLMtuPdDOX4SQMsFVQfVJ5J/mI5Xupkw/mjMRbzJ8UAIuDpdp7JTRSN2mb6l9bw0+P25Ba27itDSYf8hzWMssgseEMn06TJ/eJ4aiXTLVjdWcLD1pjai1ylpBbYG1jovWpYOsRflEEagPxFnbvW7kaxkhZQJB7yCWWRII+gKyDFmrsfy8GIzVZPnPxy3uNfzcPS/vfaqqYONG5DUpSD/sxiKp+K/7nbxW9TQ7mcANN0hWQbvItdL7PJUNlXaydrxUibAV1xfb1XNzdCNQH0iAPoBA10ACXJS9XZk+AG9n76tmgfGLRUiZQNA7XEqkTAJmAgNkWX5dkqRQIFCW5Z96pKW/ACFlgquOxjKMh55Dm/0P6h1CqRn2LgGRv+mZP/5ZWcqqAR9+COXllAbEsEr3FP+bNYsGdERHYxW04cN7LdtGj1NuKOdE6QkyKzIpqiuisK7Qfl9bSL2xvsN5WpUWf71/m6zZSJvta38Xfxw1Vy6X3OVASJlA0DtcipS9j5J+cJIsy0MkSfIEtsmy3J10iFcEIWWCqxVT4X9p3vcYOkM6lb5JuCe8h8oltGdu1tCgLC2wbBkcPozF3ZPD8Q+ztO5JNu4Pw2KBiIg2QRs16toVtK6oa66jsLZN1mxf2+5L60s7XRzcy9nLXtz0gdaIm63AuTu694nom5AygaB3uBQpOyTL8khJkn6WZXlES9kRWZa7WlzkiiOkTHA1I5uaKDn0F3wylyCr1MjDXkc7+CnoqTxdsqzk2Fi+HDZtAlmm6dYkvhv8FMtPTOL7HRImE4SGwm9+A3ffDQkJPb5e+lWF0Wyk1FBKYW2hXaStvcAV1hZaE+ja4qh2xNPZEw8nD9wd3ZW9k3vba0d33J3cuzzu6uh6WSYwXO9SVlRUpJ44cWIUQFlZmValUsleXl4mgMOHD5+yTW/x2muv+S1cuLDM1dX1vGskjB49Ouqtt97KHT9+fLdyiU2fPj1s6tSp1XPnzq28lGfp7XsILo5LWZDcKEmSGiU3NJIk+SIW7hAILhuSxhH/0a9SGHovqoML8D+8CFN2CpoxK8G7BwLSkgQ33aRseXnwwQc4rlzJ1G82M3XoUOqXLOBf+tl8utWFDz6A//t/ISAApk1TImgTJoDmOs/rqlVrrQlzz4csy1Q3VXcQtqK6IiobKqluqqaqsYrKxkqyq7KpaqyiuqmaRtP5F8KUkHBzdLMXuXZS16Gs3WtnjZjoEBAQYD59+vRJgEWLFgXp9Xrza6+9VtxZ3ZUrV/r/9re/rbiQlAkEl0J3PlqXAV8AfpIkvQHcDbzUo60SCK5DAgOjqbr5G34+9g+GnPsj6m/HIA16AmLfUFYP6AlCQuBPf4KXXoJPP4Vly3B59nFmevwPM+fNo+5PT7L1ZDibNsE//gHvv6+skX7nnYqgTZ4MDmLJ4y6RJAkPJw88nDwY4tv9GRVNpiaqm6qpbqy2ilpVYxXVjdWdv26qJq8mjxONJ6xlFvn87qBVieWtOmPz5s2uL774YqjZbCY2NtawZs2ac2+//bZvSUmJdsKECZGenp6mffv2pc+cObPfkSNHXBobG1VJSUmV7777bsH5rhscHDwsKSmp8vvvv3dzdHSUP/nkk6yYmJgmgJ07d+qXLVvmX1paqn399dfzWiNaf/zjH/2/+OILr+bmZumOO+6oevfddwvS0tIcfv3rXw8aPXp03YEDB/T+/v7N3377bYZer5dTU1OdH3/88f4NDQ2q/v37N61fvz7b19fXbpWBJ554Ivjbb7/1UKvV8sSJE2s+/PDDvM7aK+gdLihlsiyvkyTpIHAzSjqMu2RZPtXjLRMIrkM8nB0YMnIeB3xuJiDzDSLO/A1yNyHF/R/od2/PDfJycoIHH1QWP//xR2Xc2bJl6N99l/umTuW+p57CsHoy326T+Oc/YeNGWLUKIA4xtwAAIABJREFU3NwgOVkRtNtuA2cRfLksOGoc8dP44efi94vOl2WZemP9BaXur/z1Mrf86qaxsVH12GOPDdi2bVva8OHDm6ZNmxb25ptv+r788ssl77//vv/OnTvTAwMDTQDvvPNOvr+/v9lkMpGQkBC1b98+5zFjxjSc7/ru7u6m9PT0kytWrPB+6qmnQnfs2JEBUFxcrD1w4MDpw4cPO02bNm3g3LlzKz///HO3jIwMp6NHj56SZZnJkycP/Prrr/Xh4eHNOTk5TmvXrs1KSEg4d/vtt4evWbPG84knnqh46KGHBrz77rs5d9xxR90zzzwT9MILLwStWrXKuqZoUVGR+quvvvLMyso6rlKpKCsrE8tj9DG6lDJJktxkWa6RJMkLKAE+sTnmJctyxZVooEBwveGkUZMwYACHde+wo+AeRue9iH7PfZD1d4h/D1wjeu7mkqQMIEtIgPx8+OADWLkS/v1vdEOGMG3BAqatnEPTKj3btytD0jZvhrVrwcUFbr9dEbQ77uCqW1HgWkKSJPQOevQOeoLdus4a3LekbF4oHNdd3mvGGGBVtxc6N5vNhISENA0fPrwJ4KGHHip/7733/FD+Btrxj3/8w2v16tU+JpNJKi0t1R45csTpQlL24IMPVgD89re/rXjppZesM3qSk5Or1Go1cXFxjeXl5VqAb775xm3Xrl1u0dHR0QAGg0F1+vRpp/Dw8Obg4OCmhISEBoARI0YYsrOzHcvLy9W1tbXqO+64o67lHuX33HOPXUZCb29vs6Ojo2XGjBlhU6dOrZoxY0Z1d382givD+UaKrm/ZHwQO2Gyt7wUCQQ+hkiRG+LvRf0Ai30Vu5XTYG8ilqfBVDBx/AzoZPH7ZCQ6G11+H3FxYs0axriefhOBgHF9cyB1RGaxapazR+d13MHs27NoF990HPj5w112QkqKkTBMIriVOnz7tsGLFCv+dO3emp6enn5w0aVJ1Y2PjBWdeqGxmy0iSZJ1EYDuhoHXynSzLPPPMM4WnT58+efr06ZM5OTnHFy5cWAbg4OBgra9Wq2WTydStELpWq+Xw4cOn7r777sqtW7d6TJw4cVB3zhNcObqMlMmyPLVlP+DKNUcgELQiSRIRni64OWrYp5lHrvuvSSz9E05HX4LstTDqA/Cf0PMNcXRUjGvWLNi3T+naXLFCmQFw++1on3qKybfcwuTJKlasUCZ2btoEn3+uRNG0Wrj5ZiWCdtddirAJBB3pfkSrp1Cr1eTn5zscP37cMSYmpmnNmv/P3p2HRVm1fwD/nplhG2bYhkV2EBhgQAlBFgW31PBNsNzQLLefolSamkv5llkWlWb10qbllmUumfuaW7jlAikKwyIqiIKCMAzLDMsw5/fHA4TIpoKgns91PRfMM89yQwY355znvtdLQkNDSwDA0NCwWqlU8qytraFQKPgGBgZaMzOz6uzsbMFff/1l3Ldv35KWrr9+/XqzmJiY26tXrzb19fW9vyhePUOGDClevHixTVRUVKGxsbH2+vXrOvWTsYYkEkm1kZFR9YEDB0RhYWGlq1evlgQHB5fWP0apVPJKS0t5kZGRyoEDB5a6uLh0a+33hnk8WlxTRgh5GcBRSqmy5rUJgH6U0h3tHRzDMICFUA/9Hczx9y0+9ll/i0DrV2CTOhfkSD/AeQLguwzQt2j/QAgBgoK4bflyblpzxQogLAxwdwfefBP8CRPQp48YffoAX30FnD/PJWh//AFMnQpMn849HBAZySVopm3cyIBhHoW+vr52xYoVmaNGjXKpXeg/d+7cfACYMGHC3bCwMKmVlVXl2bNn0729vVUuLi7e1tbWlX5+fqUtXRsAFAoFXyqVynR1demmTZuuNXfs8OHDi5OTk/V79uzpAQBCoVC7YcOG6wKBoMnEbO3atdejo6MdZ86cyXNwcKjYuHFjZv33i4qK+EOHDnWtqKggALBkyZIOT4SZe7WmTtlFSulzDfbV1Sxr84AIWQxgKoD8ml0LW2qGzuqUMc8CjVaL+FwlckrL4WRI8Vx+LHipywEdI8B3KdB1EvC4m29XVABbt3KjZ+fOAWIxMGkS8OabgNu/MyOUAhcvAlu2cA95Xr/OjaANHswlaMOGcQ8NMI/Xs16n7HGytbXtFh8fn1L7oADzbGuqTllrfoI3dkx7Vyn6ilL6XM3WbELGMM8KAY+HQBsTeEpEyCwjiDObi/LBCYCxDDg7BTjcFyhKfrxB6ekB48Zx05pnz3LZ1Q8/AFIpt+p//35AqwUhgK8v8OmnwNWrXP721lvA5cvcA5+WltzI2caNQGmrxhwYhmGePq1JyuIJIV8SQlxqti/BLfZnGOYxI4TA01yMQBsTFFdocLTICoW9DwGBqwGlHNj/HHDxXUDTqmLibSsggFvZf+MG8OGHwIULXGLm4cGNpBUX13wNXAunZcu4EbPTp4HoaG6q85VXAAsLrovA778Dqg74MhimPdy6desyGyVjWtKapGwGgEoAm2u2CgBvtGdQAN4khFwihKyp6bXJMEw9tmID9HOUgEcIjt9U4Ib5GGBoKuD8KiD/DNjrBdzqoEHmLl2ARYuArCxgwwau2uxbb3FPc06bxo2o1Syb4PGA4GBu/Vl2Nvf05pQpwMmTwOjR3Aja2LHA9u1AefNF7hmGYZ54La4pa5ebEnIYQJdG3vovgDMA7oJr67QEgDWldHIj14gCEAUADg4OfllZWe0XMMN0UhUaLc7lKpCvqoSbqSG8LcQgeceB89OB4lTAfiTg9zUgbLpW1WNx/jzw3Xf/Dn/JZMDkydxTnZb3F0itruYStM2buYcE7t7llqsNG8Yla4MHczOnzKNha8oYpmM8cENyQsjXlNJZhJDdqOl7WR+lNKLNo7w/BicAeyil3s0dxxb6M88yLaW4lFeMa0UqWAr1EGBjAl1UASlfAMkfA0TAPQjgFt3RoXJTmJs3c+0AzpzhmmiGh3MJWlhYo001NRrg2DHutG3bAIUCMDbmenGOHs09zanDOgY9FJaUMUzHeJikrAel9B9CSKOFkCilcW0bYt19rSmluTWfzwYQSCkd09w5LCljGOB6kQoX7yhhqMNHsK0ZxHoCoOQqcD4auH2IK53hObejw/yXXA6sXcsVps3L46Y9J0zgEjSptNFTKiuBI0e4BG3HDkCpBMzMgOHDuQStf3/WLP1BsKSMYTrGwzx9uazm438opXENt3aJkrOUEHKZEHIJQH8As9vxXgzz1HA2ESLUXoIqLcWxG3dxu7Sca8nUbz/XN/PCPCDjx44O818yGbfa/+ZNbtFYz57AF19wNc9CQ7mErcGjmLq6wJAhwLp1wJ07XHHaIUOATZu4KU0bG+6hgWPHuClQhmkJn8/38/DwkNVuaWlpunv27BH379/ftaNj69u3r2tj/SnnzJljs2jRIquOiIlpX839TWlNCOkFIIIQsglcM/I6lNJ/2iMgSulr7XFdhnkWmAt10d/RHH/fKsTpWwp4mYshNTMECf4F0JQC56YDAhHg9EpHh/ovHR2uHsZLLwG5udwTnKtXcyNmM2dyhcwmT+aeCKjXkF1Pj2uGHhEBqNVc9Y0tW7iBtxUruIG3kSO503v14h4qYJiG9PT0tKmpqfL6+65cudIpVizGxcVldHQMzOPV3I+pRQDeB2AHYHmD7Yv2D41hmIch1OGjr4M57MT6SL5bgvjcIlQTHSBkK2DZB/h7PHBzZ0eH2Thra2D+fCA19d9HMDdtAnr3Bjw9gaVLuWabDRgYcFOYmzZxM6GbN3OJ2KpV3KCbgwMweza3jK0Dnm1inmDHjh0TPvfccx6enp4yX19fj8TERD0A8PHx8YiPj9evPS4gIMD9+PHjwqaOj42NlQwePNglNDTUzdHR0Xv69Ol2teeuXLnSTCqVytzc3Lyio6PrnsqxtbXtlpubKwCABQsWdHFycvL28/Nzr580fvzxx5YuLi5eUqlUNnTo0HsakDNPnuaSslxK6RAAyyilAyil/ettAx5XgAzDPDgBj6CntQm8zMXILinH8ewCaHj6QN/dgJkfcHI0cPtIR4fZNEK4RGz1ai4JW7OGK2C2YAFgZ8cNj+3cCVRV3XeqoSGXy/3xB5egbdgA+PkB33/PDba5uACffw7k5zdyX+aZU1FRwauduhw0aJBLw/d9fHzKz58/n5qSkiL/4IMPbs2fP98OAIYPH164YcMGMwDIysrSycvL0+nTp4+qqeMBQC6XC3fs2HEtJSUledeuXaYZGRk6mZmZOosXL7b966+/0uVyefKFCxcMf/nlF5P6MZw4cUK4fft2s8uXL8sPHTp0JTEx0bD2vdjY2C5JSUny9PR0+bp161gZgidcc9OXsQD8ALwE4KPHEw7DMG2FEAJ3iQgiXT7O5hThwu0i+FubgPTbz1X/Pz4M6H8IsAju6FCbJxJxrZsmTQLS0ri1Zj//DOzezZXTGD+em9709LzvVLGYK0j7yivcQwE7d3L53TvvcKXURo0CXn/9vplRpgNMnjzZPikpSdiW1/T29latWdN8o/PGpi/rKyws5EdGRjpnZmbqE0JoVVUVAYDx48crBg0aJP3qq69y1q9fbxoeHq5o7ngACAkJKZZIJNUA4OrqWn716lW9/Px8QVBQUImNjY0GACIjIwvj4uJEr732WlHteceOHRP95z//KRKLxVoAGDx4cN177u7u6pdfftk5IiKiaNy4cXX7mSdTcyNlVYSQHwHYEkJiG26PK0CGYR6NrdgAMnMRskvKcVWhAvTMgAGHAH1r4K//AIqLHR1i67m7A599xlWa3b2bG037+mvuoYHa+cqazgENGRtz+dtffwFJSUBU1L+XeO45rr86a/HENLRgwQLbvn37lly5ciV59+7dGZWVlTwAcHZ2rjIxMdGcPXvWYNu2bWavvvpqYXPHA4Curm7d5Dmfz78nYXtYx44du/LGG2/k//PPP0JfX1/PqkZGj5knR3MjZUMBDATwAlhbJYZ5ormbiaAor8Ll/GIY6wtgIewCPH8YOBQKHB0MDDoBGLl3dJitJxAAQ4dy2507wK+/clOdU6dy3QNGjeJGz0JDGx0C8/ICvvmG68X522/c1Ob06cC8eVziFh3NHcM8Pi2NaHWU4uJivp2dXSUArFy50rz+eyNGjCiMiYnpUlJSwg8MDFS3dHxjQkNDy+bPn2+fm5srsLCw0Pz+++9mr7/+el79YwYMGFA6efJkp48//ji3qqqKHDp0yGTChAn51dXVuHr1qm54eHjJ4MGDS+3t7c2USiXf3NycPXv8hGpypIxSepdSuglABKX054bbY4yRYZhHRAiBfxcTiHT5OJdTBFVVNWDoCAw4zCUtRwcCpZkdHebDsbIC3n4bSE7mVvK/+ipXZbZvX67e2aefArduNXqqSMSNmF24wPXgfOkl4KefAG9v7vTNm7naaMyza8GCBbcXL15s5+npKdNo7m1d+eqrryr27t1rNmzYsMLWHN8YR0fHqg8++OBW3759pZ6enl4+Pj5lr7766j3TkCEhIaqXX3650Nvb22vgwIFu3bt3LwMAjUZDXnnlFWepVCrz9vaWTZkyJY8lZE+2FtssEUKkAH4AYEUp9SaEdAeXqH38OAJsDVY8lmFap6RCg2M37kKkK0Bfewn4PAIoLnFrzPQk3IiZgXVHh/noysq4lf5r1gBxcVw9jLAwbvQsPJwreNaEu3e5ZWs//MA1TLey4vpxRkVxT3E+TVjxWIbpGA9TPLbWTwDeBVAFAJTSSwCarbDPMEznJNYTwL+LCYrKq3DxjhKUUsC0O9B/P1B+Gzg6CKgo6OgwH52h4b8LyK5cAd59F0hM5AqXWVtz05yHD3M9nBowN+emMTMyuNpnAQHcYJuzM9d788ABQKt9/F8SwzBPv9YkZUJK6bkG+1oek2UYplOyEevDXSJCVrEa14tU3E7zIKDPLqAkAzgWBlQ1vlj+ieTqCnz8MZCVBezb928LgEGDuBYAr7/OJW8NWgDUDq7t2gVcu8Y9sXnmDHe6VMo1I7jLxngYhmlDrUnK7hJCXFDTlJwQMhJAbrtGxTBMu5JJRLAy1ENiXjEK1DWLproMAEJ+557GjAsHNKqODbKt8flcRvXrr1wBsz/+4Jpl/vwz99HenusgcOrUfUNhjo7AJ59wD31u3AjY2nI1bu3suAE5VpSWYZi20Jqk7A0AKwF4EEJuAZgFYHq7RsUwTLsihCsuK9Th48wtBdSamlEiu3Ag+Bcg7wRwYiRQ/ZSucq9tAbB5878tAIKDuVX+ISFcFjZnDnD27D3Zlq4uMGYMt0zt8mVurdmOHdypPXpwp5eVdeDXxTDME63FpIxSeo1SOhCABQAPSmkIpZRVDWaYJ5wun4cgW1NotBRnbymgrU0+nMYAAT8CufuB0+MA7VO+WqFhC4BffwV8fYHvvgOCgoCuXblOAv/8c0+C5u0NfPst92DnihXc4FpUFDcjOnMmkJLSgV8TwzBPpBaTMkKIMSHkSwBxAI4RQpYTQozbPzSGYdqbsZ4O/LoYo7C8Col59daRuU4BenwJZG8Fzk0F6DOysl0sBsaN4xaS3bkDrFvHdQr48kuuV5NUCrz3HnDpUl2CJhYD06YBFy9yM58REVwhWpmMmxXdsoWV1WAYpnVaM325BkAJgNE1WzGAte0ZFMMwj4+dkQHcTA1xvUiFTGW9dWQes4Fui4Fr64CE2c/eoikTE2DCBO7hgNu3ublJJyfuUUwfHy7rWry4bkiMEK6pwC+/ADdvcv01s7KAyEhuNvT997k1aUznQgjxmzp1al1/ykWLFlnNmTPH5mGvN23aNDtXV1evadOm2c2ZM8dm0aJFVm0T6cPJzMzUCQsLa7RReW0T9dZea8SIEU5r1641bbvoOuYenVlzFf1ruVBKR9R7/SEh5Anqy8IwTEu8LMQoquDKZBjpCmBmUFPHy3sR9yRm6peAjhHgs6RjA+0oEgm3gGzKFG6Kc9s2bh3aRx8BH34IdOvGTYFGRgJubrCw4B4EmDsXOHiQ6xjwySdATAxXJm3aNC5R02q5hz5rP9b/vLl9D3p8U9dguNZH+/btM83Nzb1tbW39yHP1v/32m7lCobgoEAjwKMldW3Fycqo6cODAtY6Og2md1iRlakJICKX0JAAQQnoDULdvWAzDPE48QhBgbYpjWXdxNkeB/o7m0BfwueEf3y+4xCz5Y0BHDMjmd3S4HcvSkuvJNH06kJsLbN3KJWjvv89tvr5ccjZ6NHjOzhgyhHvoMzMT+PFHrj3nzp0d/UUwtfh8Ph0/fnx+TEyM1TfffHNP64e0tDTdCRMmOBUWFgokEolm/fr1mW5ubpUjRoxwEovF1YmJiYb5+fk6S5YsuTlp0iTFgAEDXFUqFd/b21v29ttv31OlYPny5eZr1661qKqqIk5OThVbt269XllZSbp16ybLzs6+zOfzUVxczJNKpd5ZWVmXv/32W0nD48Visbape2u1WkRHR9sdPXrUmBBC582blzt16lRFWlqa7tChQ92uXLmSXFpaSsaMGeMsl8sNXFxcysvLyxvtvWlra9stPDxccfToUSM9PT26cePGa97e3hUAEBcXJ4qNjbWqf28AeP/99622b99uVllZSV588cWir776KictLU13yJAhbgEBAaXx8fEiKyuryoMHD2aIRCJ6+vRpg+joaEe1Ws1zdHSs+O233zItLCzuqUvz+uuv2x48eNCEz+fTfv36Ff/444832/a/fufTmunLaADfEUIyCSGZAL4Fe/qSYZ46egJu4X9FtRbncor+XfhPCNBzBeAQCVxcAFxZ0bGBdibW1sCMGcDJk9zc5Jdfco9ovvMO94BAQACwfDmQnQ0nJ26kLDsb2LOHK5X2++/coNuOHVxz9L17ueK0f/4JHDnClU87fpxbq/b338C5c0BCAtcW6tIlrrF6SgqQlsYVu71+nZsyvXkTyMnhlsXl5wOFhYBSCZSUACoVUF4OVFU+Y9PRzZg3b17etm3bzAoKCvj190dHRzuMGzeuID09XR4ZGVkQHR1tX/venTt3dOLj41N37tx55YMPPrAFgKNHj2bo6elpU1NT5VOnTlXUv9a4ceMUSUlJKWlpaXJ3d3d1bGysuUQiqfb09FTt27dPDACbN2827tu3r1JPT482dnxz916/fr3J5cuXDVJSUpKPHDmSvmjRIrusrCyd+jF88cUXlgYGBtpr164lf/zxxzlyudywqe+JsbGxJj09XT5t2rS8GTNmNPt1b9u2zSgjI0P/0qVLKSkpKfKLFy8K9+/fLwKAGzdu6M+cOTMvIyMj2djYuHr9+vWmADBx4kTnmJiYm+np6XIvLy/1ggUL7hlVvH37Nn/fvn2mV65cSU5PT5fHxMQ8E6W4Whwpo5ReBOBDCDGqef0UVZVkGKY+E30d9LAyRvxtJZLyS9Dd0oh7g8cHev0CaMqA868DAhHg/GrHBtvZ2NkBs2dzW2Ymt8J/82ZuDnPuXG7BWWQk9EaOxIsvtmJWq6qKq6+hUv37UaUClPU+b+z95vY1/LwzSbtujzJ1q9c3tYqhgQruzi2u5DMzM9OOGjWq4LPPPrM0MDCom9i9cOGC4f79+68CQHR0dOGHH35Yt/YsIiKiiM/nw8/Pr7ygoECnsevWl5CQYLBo0SLbkpISfllZGb9v375KABg1apRi48aNpuHh4SVbtmwxe/311/ObO76pe584cUI8evToQoFAAHt7e01gYGDpyZMnhf7+/nUzWydPnhTNnDkzDwACAwPVUqm0yX8EEyZMKASAqVOnFr733nt1SVlj9z5w4IDR8ePHjWQymQwAVCoVLzU1Vb9r166Vtra2Fb169VIDgK+vryozM1OvoKCAX1JSwn/xxRdLa+5RMGrUqHvWvUkkkmo9PT1tZGSk09ChQ4siIyOVeAa0mJQRQmIALKWUFtW8NgXwNqX0vfYOjmGYx8/BWAhFeRUyFGUw1deBvZEB9wZPBwjZAsS9CJyZyE1l2g3r0Fg7LScnblHZ/PncEFZtgvbWW8CsWVyCZmbWfALVimbW9yAEEAq5zdDw3s8tLO7fJxQCS57RNYKNePfdd+/06NFDNmbMmFb1adDX168bamyphzQAREVFOW/dujUjODhYHRsbK4mLixMDwNixY4uWLFlie+fOHX5SUpIwPDy8uLnjH+beD4PH+3cijRBSd5PG7k0pxaxZs3LnzZt3z/cuLS1NV1dXt+54Pp9P1Wp1a2booKOjg4sXL6bs2rXLaOvWraY//PCD5ZkzZ9If/it6MrRmTdkQSunC2heUUgUh5D8AWFLGME+pbpZGUFZo8M/tIoh1BTDRrxkIEBgAfXZyPTJPjgb67QW6DOzYYDs7V1dg4UJuS03lkrO9e7k5RqEQMDICunS5N1lqLIFq6X19fS4xexCdKSlrxYhWe7KysqoODw9X/Pbbb+Zjx44tAABfX9+yVatWmb7xxhuFK1euNPP39y992OurVCqeg4NDVUVFBdm0aZOZtbV1FQAYGxtru3fvXjZt2jSH559/XikQCJo9vil9+vQp+emnnyzefPPNgry8PMG5c+dEsbGx2fWToJCQkNINGzaYRURElJw/f14/PT29yZHJ9evXm8XExNxevXq1qa+vb7MlkYcMGVK8ePFim6ioqEJjY2Pt9evXdeonYw1JJJJqIyOj6gMHDojCwsJKV69eLQkODr7ne6tUKnmlpaW8yMhI5cCBA0tdXFy6NRfD06I1SRmfEKJHKa0AAEKIAQC99g2LYZiOxCMEATYmOJp1F2dqFv7r8Wt+tuuIgX77gCP9gbhhwIBDgEWvjg34SeHhAXzwAbcxnc5///vf2z///LNF7esVK1bcGD9+vNP//ve/LrUL/R/22u+8805OQECAp5mZmaZHjx6lpaWldevXRo8erZg8eXLXPXv2pLXm+Ma89tprRadPnxZ5enp6EULohx9+eNPBwUGTlpamW3vM3Llz88aMGePctWtXL1dX13KZTNZksqVQKPhSqVSmq6tLN23a1OzTm8OHDy9OTk7W79mzpwcACIVC7YYNG64LBIImE7O1a9dej46Odpw5cybPwcGhYuPGjZn13y8qKuIPHTrUtaKiggDAkiVLnomCMqSloU9CyAIA4fi3NtkkALsopUvbObZW8/f3p/Hx8R0dBsM8dQrVlTieXQBzA130tjMDqT8So74DHA4FyvOA548BZr4dFyjzUAghCZRS/466f2JiYqaPjw9r697J2NradouPj09pixIhTOMSExPNfXx8nBrub02bpc8BfAzAs2Zb0pkSMoZh2o+ZgS58LI2Rp6pE8t2Se980sAIGHAZ0jIFjgwEl6yvEMAzzKFozfQlK6QEAB9o5FoZhOiFnEyGKyquQXsgt/LcVG/z7pqEDl5gdDuXWmQ06AYicOy5YhmEe2a1bty53dAzPqlY9BdHWCCGjCCHJhBAtIcS/wXvvEkIyCCFphJAXOiI+hmHu1d3SCGb6OojPVaK4osF6YyM3bl1ZtQo4OhBQ5XRMkAzDME+4DknKACQBGA7geP2dhBAZgDEAvACEAfieENLs4kaGYdofn0cQaGsKAY/gzC0FKqsb9Ogx6Qb0O8CtLzs2CChny4QYhmEeVItJGSHkrdbsexCU0hRKaVojbw0DsIlSWkEpvQ4gA0DAo9yLYZi2YSDgI9DGFGVV1YjPLbq/PpJ5ANB3N1B6DfgrjGvNxDAMw7Raa0bKJjSyb2Ibx1HLFkD9x15v1uxjGKYTMBfqorulEW6XVSCloJGSTVb9gJCtgCIR+GsooOlkVeMZhmE6sSaTMkLIWELIbgDOhJBd9bZjAApbujAh5DAhJKmRrU1KgBNCoggh8YSQ+Pz8/La4JMMwrdDVRAgHIwOkFpQip7T8/gNsXwR6bQDungJODAeqKx5/kAzTSnw+38/Dw0Pm5ubmNWTIkK4lJSXtuqzno48+sqx/D6FQ2KlqycyaNcsmjIfNAAAgAElEQVRmx44d4paPZNpDc//4TgNYDiC15mPt9jaAFhfgU0oHUkq9G9l2NnPaLQD29V7b1exr7Po/Ukr9KaX+FhYWjR3CMEw7IITA18oYJno6iM8tQkllI6WMHEcDAT8BuQeB0+MALSt3xHROtQ3Er1y5kqyjo0OXL19+zy+UqqpmC+k/sJUrV1qVlpZ21HruFn399dc5L730UknLRzLtocl/GJTSLErpX5TSYEppXL3tH0ppe/2E3QVgDCFEjxDiDMANwLl2uhfDMA+JzyMIsjUFj3AL/6u02vsPcpkM9PgKyP4DODsFoI0cwzCdSEhISGlGRobenj17xH5+fu4DBgxwdXNz81apVGTkyJFOUqlU5unpKdu9e7cY4Ho7+vn5uctkMk+ZTOZ56NAhQwDYs2ePOCAgwD0sLKyrs7OzV0REhLNWq8XHH39smZeXp9O3b19pYGCgtPa+M2bMsHV3d5f5+Ph4ZGdnC2qvHRQUJJVKpbLg4GDplStXdAEgOztbMGjQIBd3d3eZu7u77NChQ4azZs2y+eijjyzrX2/JkiWWSqWSFxwcLJXJZJ5SqVT266+/mtReu2vXrl5jxoxxdHV19erdu7dbaWkpAYARI0Y4rV271hTgisjOnj3bpvb8Cxcu6APA3r17RR4eHjIPDw+Zp6enTKFQdNok80nTmoX+wwkhVwghSkJIMSGkhBDySCt4CSEvE0JuAggGsJcQchAAKKXJALYAkIOri/YGpbT6Ue7FMEz7EOrwEWBtgpJKDRJylY03RvaYBXT7CLj+M5DwFtBOzZMZ5lFVVVXh4MGDRt26dVMDgFwuF37//fc3MjMzkz7//HNLQgjS09Plv/3227WoqCgnlUpFbGxsNCdOnEiXy+UpmzdvvjZ79myH2uulpKQYfPfdd9kZGRnJN27c0Dt06JDovffey7O0tKyKi4tLP3v2bDoAqNVqXnBwcGlaWpo8ODi49JtvvrEAgOjoaIdx48YVpKenyyMjIwuio6PtAWD69OkOoaGhJWlpafLk5GR5jx49yqOjo+9u2rRJAgDV1dXYsWOH6dSpUwuEQqF27969GXK5PCUuLi594cKFdtqaP6Bu3LihP3PmzLyMjIxkY2Pj6vXr15s29n0xNzfXyOXylMmTJ+d/9tlnVgCwfPnyLrGxsVmpqanyM2fOpIpEIvYXVxtpTfHYpQDCKaVtVq6bUrodwPYm3vsEwCdtdS+GYdqPpaEeulmIcTm/BOmFZXCXiO4/yPs9QFMMpHwBVJcD/t8CfNY+l7nXobJD9gXVBU02yH4YEr5ENchwULM9EysqKngeHh4yAAgMDCx566237h4+fFjUvXv3Mg8Pj0oAOH36tGjGjBl5AODr61tuY2NTefnyZX1XV9fK//u//3OUy+UGPB4PWVlZdf+wu3XrVubi4lIFAF5eXqqrV6/qNnZ/HR0dOmbMGCUA+Pn5lR0+fNgIAC5cuGC4f//+qwAQHR1d+OGHH9rVxCLeunXrdQAQCASQSCTVEomk2sTERHPq1CmD3NxcHS8vL1WXLl2qKyoqyKxZs+zOnDkj4vF4yMvL071586YAAGxtbSt69eqlrvmaVJmZmY3+T/nKK68oACAgIEC1a9cuUwAICgoqnTt3rv3o0aMLx44dq3BxcWFJWRtpTVJ2py0TMoZhni6upoZQlFch+W4JTPR1YGXY4Gc7IcBzSwGeLpAcAyiTgdA/AAPrjgmYYeqpXVPWcL9QKGwx0fjkk0+sLC0tq/7444/rWq0WBgYGfvWuWzcszOfzodFoSGPXEAgElMfj1X7e5HEtmTRp0t1Vq1aZ5+Xl6UyaNKkAAFauXGlWUFAguHz5coqenh61tbXtplareQCgq6tbPz5au78hfX19WhtnbWwxMTG3X3rpJeXOnTuNQ0NDPfbu3XvF19e3kad+mAfVZFJGCBle82k8IWQzgB0A6h6jopRua+fYGIZ5AhBC0KOLMUoqNTiXo8AAR3MY6goaHgT4fAKYPgf8PRE44AeEbgPMgzokZqbzaWlEqyP17t279NdffzWLiIgouXTpkl5ubq5u9+7dy3/88Ue+nZ1dJZ/Px7fffiuprm55tY2hoWG1UqnkWVs3/0eJr69v2apVq0zfeOONwpUrV5r5+/uX1sRSsmzZMotFixblaTQaKJVKvkQiqX7ttdeKPvnkE1uNRkNGjBhxDQCUSiXf3Ny8Sk9Pj+7evVuck5PT6Gjdg0pOTtYLCAhQBwQEqBMSEoRJSUn6LClrG82tKQuv2YwAqAAMrrdvaPuHxjDMk0LA4yHQhluSciZHAU1jC/8BwGEU8MIZgKcPHO4LZKx6jFEyzMOZP39+nlarJVKpVBYZGemycuXKTAMDAzpr1qy8jRs3Stzd3WWpqan6BgYGLY6uTZgw4W5YWNg9C/0bs2LFihu//PKLuVQqlW3cuFHy/fffZwPADz/8cCMuLk4slUpl3t7edYvv9fX1aa9evYojIiIKBQLuj6IpU6YUJiYmGkqlUtnPP/8scXZ2bpPEaenSpZZubm5eUqlUpqOjQ0eOHKlsi+syAGl0ce4Txt/fn8bHx3d0GAzzzLtdVo7TNxWwE+ujp7UJCGliJqaiEDg1Brh9CHCLBnp8DfDb5I945gEQQhIopf4tH9k+EhMTM318fFhPrjZQXV0NLy8v2e+//361W7durDhgJ5eYmGju4+Pj1HB/i2vKCCGxjexWAohvoeYYwzDPmC6G+pCZiyG/WwJT/TK4mTWy8B8A9MyAfvuAxIVAyjKg6DLXCcDA6vEGzDBPgYSEBP1hw4a5DRkyRMESsidbaxb66wPwAPB7zesRAK4D8CGE9KeUzmqv4BiGefK4mxmiqLwKSfncwn8LYRNPWvIEgO9SwLQHcHYyt86sz3ZA0vPxBswwTzg/P7/ymzdvXu7oOJhH15qCb90B9KeUfkMp/QbAQHBJ2svg1pkxDMPUIYTAz9oYIl0BzuYUQVXVwuJnpzHA4NMATwc4FApcW/dY4mQYhulsWpOUmQKoPwdhCMCspqgrGyZlGOY+OjwegmxNoaUUZ24pUK1tYe2q6XPAC+cBi97AmUlA/ExA27btbRiGYTq71iRlSwFcJISsJYSsA3ABwDJCiCGAw+0ZHMMwTy6xrgD+1iYoqqjChTtNVPyvT98c6H8QcJ8NpH8DHB0ElOc/nmAZhmE6gRaTMkrpagC9wNUp2w4ghFK6ilJaRimd194BMgzz5LIR6cNDIsKNYjWuFalaPoEnAPy+BIJ/AQrOAgf8gcJ/2j9QhmGYTqDJpIwQ4lHzsQcAawDZNVuXmn0MwzAt8pSI0MVQD5fyipGvauWKB+dXgUGnAFDgUG/g+q/tGiPz7Lpx44Zg6NChXe3t7b29vLw8+/bt63rp0qUm+4ClpaXpurm5eT3KPX19fT1aOuajjz6yLCkpadNG35mZmTphYWFdAeD06dMGmzdvNm7L6zOPrrn/4HNqPi5vZPuineNiGOYpQQiBv7UJRLoC/H1TgUJ1ZetONOsBhMUDkkDg79eAhDmAVtO+wTLPFK1Wi4iICNc+ffqUZGdnJyUnJ6d89tlnt3JycnTa6h5VVfevjbxw4UJqS+etXLnSqrS0tE2TMicnp6oDBw5cA4D4+Hjh3r17WVLWyTT5H5xSGlXzsX8j24DHFyLDME86XT4PIfZm0BPwcOpmIYrKW7mIX98SGHAIkM4A0r4Cjr0AlLNao0zb2LNnj1ggEND58+fXLV4MDg5Wh4WFlWq1WkybNs2utnL9Tz/9ZNrwfJVKRUaOHOkklUplnp6est27d4sBIDY2VjJgwADXoKAgaa9evdwbnicUCn1r7x8QEOAeFhbW1dnZ2SsiIsJZq9Xi448/tszLy9Pp27dvXeX/bdu2GT333HMeMpnMc8iQIV2VSiUPAGxtbbvNnj3bRiaTeUql0roK/3v37hV5eHjIPDw8ZJ6enjKFQsGrHeUrLy8nn376qc3u3btNPTw8ZD/99JOpo6Ojd05OjgDgCtE6ODjUvWYenxazcEKIkBDyHiHkx5rXboQQ1maJYZgHYiDgI9TeDAIeDydvFqK4opWJGU8H8I8FgtYC+aeAg/6A4mL7Bss8Ey5dumTg4+PT6GLH9evXm1y+fNkgJSUl+ciRI+mLFi2yy8rKumcE7fPPP7ckhCA9PV3+22+/XYuKinJSqVQEAJKTk4U7d+68ev78+bTmYkhJSTH47rvvsjMyMpJv3Lihd+jQIdF7772XZ2lpWRUXF5d+9uzZ9NzcXEFMTIz18ePH0+VyeUqPHj1US5Ysqau0bG5urpHL5SmTJ0/O/+yzz6wAYPny5V1iY2OzUlNT5WfOnEkViUR1LaD09fXpu+++mxMeHq5ITU2VT506VTFy5MiCVatWmQHAzp07jTw9PdU2NjZsaPoxa00WvBZAArjF/gBwC1wh2T3tFRTDME8noY4AofZmOH6jACeyC9HXQQJRw+blTek6ETD2Ao6/DPzZCwhcw9U4Y54KCblF9sWVGmFbXtNIV6DyszZ5qEbnJ06cEI8ePbpQIBDA3t5eExgYWHry5Emhv7+/uvaY06dPi2bMmJEHAL6+vuU2NjaVly9f1geA0NDQYisrqxY7lHfr1q3MxcWlCgC8vLxUV69eva/f2F9//WV49epV/YCAAA8AqKqqIn5+fqW177/yyisKAAgICFDt2rXLFACCgoJK586daz969OjCsWPHKlxcXJrtyxkdHX03IiLCddGiRXlr1qwxnzhxIhuS7gCtma92oZQuBVAFAJRSFYAmGtoxDMM0T6QrQIi9GQDgRHYByqoe4I9xSU8gLAEw8wNOjwUuzAe0Lf7eY5hGdevWTZ2YmNimiWAtoVDYYnNyANDT06urFcPn86HRaO77/UopRUhISHFqaqo8NTVVfvXq1eQtW7Zk1b6vr69PAUAgENDa82NiYm6vWrUqS61W80JDQz1qpzWb4urqWmVubq7ZtWuX+OLFi4ajRo1iTcY7QGv+RK0khBgAoABACHEBKxrLMMwjMNLTQW87M5zILsDJ7EL0sZfAQIffupMNrIABR4B/ZnN9MxUXgd6buH6azBPrYUe0HkV4eHjJ+++/T7744gvzuXPn3gWAs2fPGigUCn6fPn1KfvrpJ4s333yzIC8vT3Du3DlRbGxstlqtrhvM6N27d+mvv/5qFhERUXLp0iW93Nxc3e7du5efPXv2kRM9Q0PDaqVSybO2tka/fv3K3n77bYekpCQ9b2/viuLiYl5mZqZO9+7dm/xdnJycrBcQEKAOCAhQJyQkCJOSkvQDAgLqpmqNjIyqGz5IMHny5PwpU6Y4jxgxokAgYMvJOkJrRsoWAzgAwJ4QsgHAEQDz2zMohmGefib6XGJWUa3FyZsFKNc8wIgXXxfo+R0Q8BOQFwcc7Mk1NWeYB8Dj8bBr166rR48eNbK3t/d2dXX1WrBgga2trW3Va6+9VuTl5aX29PT06tevn/TDDz+86eDgcM+w7vz58/O0Wi2RSqWyyMhIl5UrV2YaGBi0UCW5dSZMmHA3LCxMGhgYKLWxsdGsXLkyc8yYMV2lUqnM39/fo3aatClLly61rH1IQUdHh44cOfKeka8hQ4aUpKenG9Qu9AeAsWPHKlUqFT8qKqqgLb4G5sGRFqtsAyCESAAEgZu2PEMp7VRzzf7+/jQ+Pr6jw2AY5iHcVVXi1M0CiHQFCLWXQJf/gFUA7p4BTgwHqoqBoHWAw8h2ifNpRAhJoJT6d9T9ExMTM318fDrV75Nn2fHjx4WzZ8+2T0hIaPbhBObRJSYmmvv4+Dg13N+apy9/BTAcwFVK6Z7OlpAxDPNkMxfqItjWDCWVGpy8WYiq6lYtxal3gSBunZlJd+DkKODiQrbOjGEe0MKFC7uMGTPGJSYm5lZHx/Isa82fpKvBVfT/hhByjRDyByHkrXaOi2GYZ4iloR4CbUyhLK/C6VuF0GgfMDEzsAaePwa4RgHyT4G4cKBS0T7BMsxTKCYm5nZOTs7lF154obTlo5n20prel8cAfALgfQA/AfAHEN3OcTEM84yxFukjwMYEBeoq/H1LgWrtAy7N4esBASuBniuAO4eBAwFAUXL7BMswDNMOWjN9eQTAKQCRANIA9KSUtti3i2EY5kHZig3g38UY+apKnMlRQNuKNa/3cZvGjZppSoA/g4Ds7W0fKMMwTDtozfTlJQCVALwBdAfgXVMig2EYps05GAvha2WMO2UVOJdT9HCJmUVvbp2ZsYx7CODSIoA+4JQowzDMY9aa6cvZlNI+4Bb7F4Cr8F/0KDclhIwihCQTQrSEEP96+50IIWpCyMWabcWj3IdhmCeTs4kQ3S2NkFNajoTcIrTmKfH7CG2BgXFA18lA0hIgbhhQyephMgzTebVm+vJNQshmABcADAOwBsCQR7xvErgk73gj712llD5Xs01/xPswDPOEcjU1hJe5GNkl5bhwR/lwiRlfHwhcBfh/C+QeAP4MBJSpbR8s88SqbQ5eKzY2VjJ+/HiHjoqHeba1pmSvPoAvASRQStukOSmlNAUACGHdmhiGaZq7RIRqSpFaUAo+IehuafTgPzcIAaRvACbdgBMjgQM9AO9FgMccrggtwzBMJ9Ga6csvKKVn2yohawVnQsgFQkgcIST0Md2TYZhOylMigpupIa4WqZB8t+ThRswAwLIPMOQCYB0GJL4L7H8OuBPXtsEyT5URI0Y4rV271rT2df1Rtffff9/K29vbUyqVymbPnm3TMREyT5t2a25FCDkMoEsjb/2XUrqzidNyAThQSgsIIX4AdhBCvCilxY1cPwpAFAA4OLCRZoZ5WhFC4G0hhoZSpBeWgU8IPM3FD3cxoS3QZxtwaw8QPwM40g9wHg/4LgP0Lds0bubJUFFRwfPw8JDVvlYqlfxBgwY1u/hw27ZtRhkZGfqXLl1KoZRi4MCBrvv37xcNGTKE1fhiHkm7JWWU0oEPcU4FapqdU0oTCCFXAUgB3NdDiVL6I4AfAa7N0qNFyzBMZ0YIwXOWRqjWUqQUlILPI5CaiR7+grZDAasBQPInXFPzm7uA5z7lis+SB2zzxLSJnTt32ufl5T1yI+/6LC0tVcOGDWu20bmenp42NTVVXvs6NjZWEh8fb9jcOQcOHDA6fvy4kUwmkwGASqXipaam6rOkjHlUnaoNPCHEAkAhpbSaENIVgBuAax0cFsMwnQAhBH5djKGlFEn5JeATAhfTZn93Nk8gBHw+AZxeBeLfAM5HA9fWcsVnzXxbPp956gkEAlpdzbXsqq6uRlVVFQEASilmzZqVO2/ePNZ2kGlTHZKUEUJeBvANAAsAewkhFymlLwDoA+AjQkgVAC2A6ZTSwo6IkWGYzocQAn9rE1RTBRLzisHnETgZP+LgirEnMOAIkPkbcGEOcNAfcHsT6P4RoGvcNoEzLWppRKsjODo6ViYkJAinTJmi+O2330w0Gg0BgCFDhhQvXrzYJioqqtDY2Fh7/fp1HV1dXWpra/u41l4zT6kOScoopdsB3Fdmm1L6B4A/Hn9EDMM8KXiEIMDaFH/fKsQ/t5XgEwJ7o0esZ00I4DwOsH0RSPwvkP4NkP074Psl4BjJvc88c2bMmJE/dOhQV3d3d9mAAQOUBgYGWgAYPnx4cXJysn7Pnj09AEAoFGo3bNhwnSVlzKMiD/0kUyfi7+9P4+PvW3bGMMxTTKOlOH2zEAXqSgTYmMJWrN92Fy84z01nFiYAXQYC/t8DRm5td/1OghCSQCn1b/nI9pGYmJjp4+PDpgCZZ05iYqK5j4+PU8P9bEUrwzBPJAGPINjOFKb6OjiXo8DtsvK2u7ikJzD4LFd0tuAcsM8buPQBUN2G92AYhmmAJWUMwzyxdHg89LIzg5GeAGduKZCvqmi7i/P4XNHZoWmA/Ugg6SNgrzeQc7Dt7sEwDFMPS8oYhnmi6fJ5CLGTQKQjwOmbChSoK9v2BgZdgN4bgAGHAcIH/goDTowCVLfa9j4MwzzzWFLGMMwTT0/AQ4i9GQwEPJy6WQhFeVXb36TL88B/LgHdlwA5e4A9HkDqV4CWre1mGKZtsKSMYZingr6AjxB7CXR5PJzKLoCyoh0SM74e4P0e8GIy17bpnznAAX8g/++2vxfDMM8clpQxDPPUEOrwEWpvBh6P4GR2IUoq22kUS9QV6LsHCP0DqCwADvUCzkYBFaysIsMwD48lZQzDPFUMdQUItZMAAE5kF6CsvRIzQgD74cCLKYDH28C1NcAed+DaOuApKDX0rLh69arO888/7+Lo6Ohtb2/vPWnSJPvy8nICAHv27BGLxeLnPD09ZU5OTt7+/v7uGzdurKsovHjxYisXFxcvqVQqCw4Olqanp+vWvjd9+nQ7V1dXr65du3pNnDjRXqvVAgBOnDghlEqlMgcHB+/6+wHgyJEjhmPGjHEEgIMHD4q6devm6ezs7OXs7Oz1xRdfmNceN2fOHBtLS8vuHh4eMkdHR+/Bgwe7JCQk1NWEGT16tKO7u7tMKpXKwsLCuiqVSh4A7N+/XySTyTwFAoFf/UbrAMDn8/08PDxkHh4esgEDBrg+yPfwwoUL+h4eHjJPT09ZcnKyXlPHxcbGSsaPH+8AAEuXLrX49ttvJcD9jd8bs2fPHvGhQ4ceuIWHra1tt9zc3AeuyRoZGelY/3v6uLCkjGGYp45YT4AQOzNotRQnbhZCVVXdfjfTEQE9vgDC/gHEUuDMJOBwH6Aoqf3uybQJrVaLl156yTUiIqIoKysr6fr160llZWW8t956y7b2GH9//9KUlBR5ZmZmUmxs7I25c+c67Ny5UwwAfn5+qosXL6akp6fLX3rpJcXs2bPtAODQoUOG586dE6Wmpianp6cnX7x40XDfvn1iAHj99dcdf/jhh6zMzMyka9eu6W/dutWo9l579uwxDgsLU964cUMwceJE5x9++CHr+vXryadPn05bu3atxaZNm+oSwunTp99JTU2VZ2VlJY0aNarwhRdecM/JyREAwIoVK7LT0tLk6enpcjs7u8rPP//cEgC6du1auXbt2szw8PCCht+L2h6gqamp8qNHj2Y8yPfx999/N4mIiFCkpKTIvby8WvUI9Pz58/PffPPN++JoytGjR8UnTpx4hIa3D2bz5s1Zfn5+j70GDkvKGIZ5Khnr66C3vRkqq7U4mV2Ack07JmYAYNodGHQCCFwNFKcA+32BC/OBKtajurPavXu3WE9PT/vWW28VAIBAIMCKFSuyN2/ebF5SUnLf78devXqp582bl/Ptt99aAkB4eHiJWCzWAkBISEhpbm6uLsC1A6uoqCDl5eVErVbzNBoNsbGxqcrKytIpLS3lPf/882U8Hg/jxo0r2LFjR90I0fHjx8Xh4eEly5cvt4yMjCwICQlRAYC1tbUmJibm5rJly7o09nVMnTpVERoaqly9erUZAJiZmWkBLulUq9U8UtORwt3dvTIwMFDN4z3cr/7Tp08b+Pj4eEilUtmgQYNc8vPz+Zs3bzb+8ccfrdatW2cRGBgobXjO//73P4mTk5N3t27dPE+fPl2XVM2ZM8dm0aJFVg2Prz+ydfz4cWFAQIB7Wlqa7vr16y1WrFhh5eHhITtw4IAoJydH8MILL7h4e3t7ent7e/7555+GAHD79m1+79693VxdXb0iIyMdGyuQv2bNGtMpU6bYAcCSJUss7ezsugGAXC7X7dGjhwcABAQEuB8/flwIAEKh0HfGjBm27u7uMh8fH4/s7GwBADQVw969e0W1o46enp4yhULR6m84S8oYhnlqmerroredGdQaLU5mF6JCo235pEdBeIDLZODFVKDrBCBlGbBXBmRvZ1OandDly5cNfHx8VPX3mZmZaa2trSvlcnmj03ABAQGqq1ev3jettXLlSouBAwcqAWDgwIFlvXv3LrG2tvaxsbHp3r9//+IePXqUZ2Vl6VhbW9c9geLo6FiZm5urAwC5ubkCgUBAJRJJdUpKioG/v/89cYWEhKgyMjKa7Cfm6+urSk1NrYtr5MiRThYWFj4ZGRn677zzTl5L34vKykqet7e3p4+Pj8cvv/xi0tgxEydOdI6JibmZnp4u9/LyUi9YsMAmMjJSOX78+Pzp06ffOXv2bHr947OysnQ+++wzm9OnT6eeP38+NT09/aH6obm7u1fW3iM1NVUeFhZWOm3aNPs5c+bcSUpKStm+ffvV6dOnOwHAO++8YxMcHFyakZGR/PLLLxfVJsr1DRw4sOTMmTNiADh16pTIxMREc/36dZ0jR46Ig4ODSxoer1arecHBwaVpaWny4ODg0m+++cYCAJqKYfny5V1iY2OzUlNT5WfOnEkViUSt/sHTIb0vGYZhHheJgS6CbU1x+lYhTt0s4J7Q5Lfz36P65kDgKqDrJK5d04nhgM2LgP83gMi5fe/9hPp68i77rKT8R+wufy9HbwvVrDURbdrovLGRl++//94sMTFRuHLlyjQASEpK0ktPT9e/efPmJQDo27ev9MCBAyKhUNjkL+edO3caDRgwoLit4tq6dWumRqPBxIkTHdasWWNaOxrYlCtXrlxydnauksvluoMGDXLv0aOHuv5UZEFBAb+kpIT/4osvlgLA1KlTC0aNGtW1uWseP37cMCgoqMTGxkYDAMOHDy9MT09vk3Vap06dMrpy5UpdkldaWspXKpW8M2fOiLdt25YBAGPGjFFOmzbtviFyBwcHjUql4ikUCl5OTo7uqFGjCv7880/xyZMnRcOHDy9qeLyOjg4dM2aMEgD8/PzKDh8+bNRcDEFBQaVz5861Hz16dOHYsWMVLi4urU7K2EgZwzBPPUtDPQTZmEJZocGpm4VQt/dUZi2L3kBYAuC7HMj7C9jrBSTHANVtXOCWeSje3t7qxMTEexLBwsJCXm5urq5MJmt0bdT58+eFrq6udWuNduzYIf7iiy+s9+3bl2FgYEABYPPmzSY9e/YsMzY21hobG2sHDoYfPk0AABJVSURBVByoPHnypKGjo2NV7cgYAGRlZenWjpwdOHDAeOjQoUoA8PDwUMfHx98T16lTp4Surq7qpr6WixcvCj09Pe9ZAyUQCDBu3LjC+lOkTXF2dq4CAJlMVhkUFFRy7ty5Nk2QW4vP59Pahx/UanWTOQqlFP/8809K7Tq4vLy8S8bGxq1Ofvz9/Uu/++47cxcXl/L+/fuXnjhxQpSQkCAaOHDgfesNBAIBrZ3yFQgE0Gg0pLkYYmJibq9atSpLrVbzQkNDPS5cuNDqRJSNlDEM80zoItJHkK0pzuUU4WjmXQTZmEIivG9mo+3xdADPOYDjaCBhFpD4X+D6esCyP6BnBuiaAbqm3MeGrwUPNdvzRGrrEa3WiIiIKHnvvfd43377reTNN98s0Gg0eP311+1HjRp1t3atWH1nz541WLZsmc3333+fCQCnTp0ymDFjhuO+ffuu2Nra1j3m6+DgULl27VqLqqqqXK1WS06dOiWeMWPGHUdHxyqRSKQ9cuSIYf/+/cs2bNggeeONN/K0Wi1SUlIMgoOD1QDw9ttv5wcFBXmMHj1a0atXL/Xt27f5CxcutFu4cGFOY1/HunXrTE6cOGH83Xff3dRqtZDL5Xre3t4VWq0W27dvN3Fzc2t2wXp+fj5fJBJpDQwMaG5uriA+Pl60cOHC2/WPkUgk1UZGRtUHDhwQhYWFla5evVoSHBzc7ILJPn36lC1YsMD+9u3bfFNTU+327dtNvby8mkwsAcDOzq7y1KlTwtGjRxdv2bKlLpkUi8XVxcXF/NrXISEhxZ9++qnlkiVL7gDcerdevXqpg4KCStatWydZunRp7pYtW4zqn1NfSEhI6aeffmozb9683F69eqkmTpwo1tfX10okklb/xdZUDMnJyXoBAQHqgIAAdUJCgjApKUnf19e3VQ8NsKSMYZhnhrVIH/0cJThzS4Hj2QXwsTKCs7EQtQuh25XQDgjdCuTsBy4tArL/ACoVAG2mZAdf/98ErTZZa5i41SVz9RI7HWNufRvTLB6Phx07dmRERUU5Llu2zFqr1WLAgAHK2NjYuh5a8fHxIk9PT5lareZJJJKqZcuW3Rg2bFgJAMybN89epVLxR40a5QIANjY2lUePHs2YNGmS4tixY0bu7u5ehBD0799f+corrygB4Lvvvsv6v//7P+fy8nLSv3//4lGjRilPnjwp9Pb2VtWOxjg6OlatWbPmelRUlFNZWRmPUkqio6Pv1F4DAFasWGG1ZcsWiVqt5kmlUvXBgwfTbGxsNNXV1Rg/frxzaWkpj1JKPD09VevWrcsCgLi4OOHo0aNdi4uL+UeOHDH55JNPbDIyMpIvXryo/8YbbzgSQkApxaxZs2439uTh2rVrr0dHRzvOnDmT5+DgULFx48bM5r6/jo6OVQsWLMgJCgryFIvF1d7e3qrmjgeARYsW5UyfPt3po48+qu7Vq1fd+q4RI0YUjRw50mX//v0mX3/99Y0ff/wxe8qUKQ5SqVRWXV1NAgMDS3r16nXjs88+yxkxYkRXV1dXL39//1Jra+tGh6Wff/750rfeekt34MCBJQKBANbW1pUtJa8NNRXD0qVLLU+fPm1ECKHu7u7qkSNHKlu+Goc0Nj/+pPH396fx8fEdHQbDME+IymotzucW4U5ZBRyNDfCcpTH4vMeQmDVEKaApBSoLucKzlQru88ZeVypq9tVsmrJmLkwAXZNGRuHuTeSIy6QESqn/Y/t6G0hMTMz08fG521H37yzmz59v7erqWh4VFaXo6FiYxyMxMdHcx8fHqeF+NlLGMMwzR5fPQy9bU8gLSpFWUIriCg0CbUwh1Gl0pqP9EALoiLnN0PHBzq2urJe0Keolcg1f13xedv3f92g7P4XKPJClS5fmdnQMTOfAkjKGYZ5JhBB4mYthqqeD+NtFOJZ1FwE2JrAQNlmQvHPh6wIGVtz2IKgWqCrhEjQ0+/AcwzCPGVt0wDDMM81GrI9+DhLo8Ll+mRmKskbLHjw1CA/QNWalORimE2JJGcMwzzwjPR30dzBHF5EeLuUVI/62EhrtU5yYMQzTKbGkjGEYBoAOn4cgG1N4SkTILlYj7sZdlFW1UzNzhmGYRrCkjGEYpgYhBJ7mYgTbmkJVVY1jWXeRV9aq/soMwzCPjCVlDMMwDViL9NHf0Rx6fD5O3ixEemHp073O7BnG5/P9PDw8ZG5ubl5Dhgzp2lgj8qbs2bNH3L9/f1cA2LBhg/HChQsbbRjOMK3FkjKGYZhGiHQF6O8oga1IH0n5JTifWwSNlpWSeNro6elpU1NT5VeuXEnW0dGhy5cvt6j/flVVVVOn3mPcuHHKmJiY2y0fyTBN65CkjBCyjBCSSgi5RAjZTggxqffeu4SQDEJIGiHkhY6Ij2EYBgAEPB4CbEzgZS7GzZJy/JVVgNJKts7saRUSElKakZGht2fPHrGfn5/7gAEDXN3c3LxVKhUZOXKkk/T/27v72KrqO47jn2/v86XPxQGFIm5aSuVhWBAT4sOcyyQxNUOzuCwhRE2j2YzxD82MMfqPMf7jH8YQNZtmmzBnDAuaqUzYCMsQI25QSkFkPCgPpaW0UOjTLf3tj3vL7UhbdmfvPefevl9/tedeDt/7Tc/pp79zfudXW1u/YMGC+g8++KDkyn/7yiuvVK1Zs2auJN13333z1q5dW7N06dK6OXPmLHrrrbcuLxf07LPPzli4cOGC2tra+ieeeKI6l58P/ufVSNknkhY65xZLOijpaUkys3pJD0i6UdLdktaZWY6f5ggAaWam+VXFWjmnUn1DyfvM2i5mtBoL8kAikdDmzZtLFy1a1CdJra2t8XXr1n199OjRlpdeeuk7ZqaDBw+2btiw4XBTU9O83t7eCZeAOH36dGjXrl0HNm3a9NVzzz03W5I2btxYeujQoWhzc/P+/fv3t+7evTv+0UcfFefi8yE/ePLwWOfcX0Z9u1PS/amv75X0jnNuQNIRMzsk6WZJn+a4RAD4LzOmRXTntdO182SXdhzvUv30Es2vnJabdTOngp0P1qi7JT6p+yxf2Ktb3pxwofOBgYGiurq6eklasWJFz+OPP35my5YtxYsXL75YV1c3KEk7duwofuyxx9olaenSpf3V1dWDe/fujU6038bGxu5AIKCGhob+zs7OkCR9/PHHpdu3by+tr6+vl6Te3t6iAwcORFetWjXhwt6YOvzwRP8HJf0x9fVsJUPaiOOpbQDguWnhoG6fW6V/tp1T65kedfcn1DCrTKEibs/NVyP3lF25PR6Pf6sbCKPR6OWZISOTRFKLfZ968sknp/x6nxhb1kKZmW2RNNZMlGecc5tS73lG0pCk9f/H/pskNUnS3Llzv0WlAPC/CxYVafmsclVEL6qlo0fbjg3pltkVKgn74W/cPHaVES0vrVy58sLbb79d2djY2NPc3Bw5depUePHixf1bt27N6NLjqlWrzj///PPVTU1NZ8vKyoaPHDkSCofDbvbs2dyoCElZDGXOubsmet3M1kq6R9IPXXqu+QlJNaPeNie1baz9vyHpDUlatmwZc9UB5IyZ6YbKYpVHQ/rsZHLdzOWzyjWreMIrWshTTz31VPuaNWuura2trQ8EAnr99dePxmKxjH/vrF69+vy+ffuiy5cvr5OSo3Hr168/QijDCPPi2TtmdreklyXd7pzrGLX9RkkblLyPrFrSVkk3OOcuTbS/ZcuWuV27dmWxYgAYW2/iknae6FL3QEJ1VcVaUFWcN/eZmdkXzrllXv3/e/bsObpkyRIu5WHK2bNnz/QlS5bMu3K7V+Ptr0qKSPokdfLa6Zx7xDm3z8zeldSq5GXNX1wtkAGAl+KhgG6fW6V/nT6nA50X1N2f0PJZ5QoFuM8MQGa8mn15/QSvvSDphRyWAwDfSqDI1DCzTBXRkJrbz+tvx87oltkVKo2EvC4NQB7hTzkAmARmpu9VTNOtNVVKDDttO9apEz19XpcFII8QygBgEk2Ph3XntdNVEgnqs5Pdauk4z7qZ4xseHh7OjxvwgEmS+pkf85ErhDIAmGSxUEC31VRpXllMB89e1D+Od2nwEutmjqGlo6OjjGCGqWJ4eNg6OjrKJLWM9ToP1gGALAgUmW6aWa6KaFi7T59L3mdWXaGyKPeZjRgaGnq4ra3t121tbQvFIAGmhmFJLUNDQw+P9SKhDACy6LryuEojQX12skvbvu7UTTPLVFMa87osX2hoaGiX1Oh1HYBf8JcJAGRZVSx5n1l5NKjPT3Vrb/t5JbicCeAKjJQBQA5EgwHdWlOl5vbz+qrror7quqjScFBVsbAqYyFVxcKaFgrkzYNnAUw+QhkA5EiRmb4/o0xzSmLq6B1QZ19C3/T06ci5XklSJFCUDGjRsCpjYVVEQwoUEdKAqYJQBgA5Nj0e1vR4WJLknNP5wSGd7Uuos29QZ/sGderCgCTJJJVHk6NoVbGQKmNhxYIBDysHkE2EMgDwkJmpLBJSWSSk68rjkqSBoUvq7E/obN+gOvsSOtx9UYe6ku+PBwOXL3dWxsIqiwRVxCVPoCAQygDAZyLBgKqLA6oujkqShp1Td38iOZrWP6gzfYM63tMvSQqYqTIWUmU0PZoWZt1NIC8RygDA54rMVJkaGbte0+ScU9/Q8OXLnZ19CR08e0Ej6waUhIOXR9OqomEVh5lAAOQDQhkA5BkzUzwUUDwUu/zMs6HhYXX1J9TZl7zseaqnX8fOJdfeDBclQ93ITM+KaFhBJhAAvkMoA4ACECwq0jXxiK6JRyQlJxBcGLykzv7B1IhaQm0XeyQlJxCwsgDgP4QyAChAZqaSSFAlkaDmlSUnEAxeGr58ufNs/6DHFQK4EqEMAKaIcKBIM4ujmpmaQADAX5iiAwAA4AOEMgAAAB8glAEAAPgAoQwAAMAHCGUAAAA+QCgDAADwAUIZAACADxDKAAAAfMCcc1d/l8+ZWY+kL72uwyemSzrjdRE+QS/S6EUavUib75wr8boIAEmF8kT/L51zy7wuwg/MbBe9SKIXafQijV6kmdkur2sAkMblSwAAAB8glAEAAPhAoYSyN7wuwEfoRRq9SKMXafQijV4APlIQN/oDAADku0IZKQMAAMhrhDIAAAAfIJQBAAD4QMGHMjO7w8z+bmavmdkdXtfjJTNbkOrDe2b2qNf1eMnMvmtmvzGz97yuxQtT/fOPxnGRxvkS8JavQ5mZvWlm7WbWcsX2u83sSzM7ZGa/uspunKQLkqKSjmer1mybjF445/Y75x6R9FNJK7NZbzZNUi8OO+ceym6luZVJXwrx84+WYS8K4rgYT4bHS0GcL4F85evZl2Z2m5IniN855xamtgUkHZT0IyVPGp9L+pmkgKQXr9jFg5LOOOeGzWyGpJedcz/PVf2TaTJ64ZxrN7NGSY9K+r1zbkOu6p9Mk9WL1L97zzl3f65qz6ZM+uKca029XjCff7RMe1EIx8V4MjxeDhTC+RLIV75eZsk5t93M5l2x+WZJh5xzhyXJzN6RdK9z7kVJ90ywuy5JkWzUmQuT1Qvn3PuS3jezP0vKy18+k/xzUTAy6Yuk1txWl1uZ9qIQjovxZHi8jPxc5PX5EshXvg5l45gt6ZtR3x+XtGK8N5vZakk/llQu6dXslpZzmfbiDkmrlTzZfpjVynIv015USXpB0lIzezoV3grRmH2ZQp9/tPF6cYcK97gYz3i9KOTzJeB7+RjKMuKc2yhpo9d1+IFzbpukbR6X4QvOuU5Jj3hdh1em+ucfjeMijfMl4C1f3+g/jhOSakZ9Pye1bSqiF2n0Ymz0JY1epNELwIfyMZR9LukGM7vOzMKSHpD0vsc1eYVepNGLsdGXNHqRRi8AH/J1KDOzP0j6VNJ8MztuZg8554Yk/VLSZkn7Jb3rnNvnZZ25QC/S6MXY6EsavUijF0D+8PUjMQAAAKYKX4+UAQAATBWEMgAAAB8glAEAAPgAoQwAAMAHCGUAAAA+QCgDAADwAUIZkCVmNtPM3jGzf5vZF2b2oZnVel0XAMCfCn7tS8ALZmaS/iTpt865B1LblkiaIemgl7UBAPyJUAZkxw8kJZxzr41scM7t8bAeAIDPcfkSyI6Fkr7wuggAQP4glAEAAPgAoQzIjn2SGrwuAgCQPwhlQHb8VVLEzJpGNpjZYjO71cOaAAA+RigDssA55yT9RNJdqUdi7JP0oqQ2bysDAPiVJX93AAAAwEuMlAEAAPgAoQwAAMAHCGUAAAA+QCgDAADwAUIZAACADxDKAAAAfIBQBgAA4AOEMgAAAB/4DzwqdvrnWHRtAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = plt.subplot(111)\n",
    "    \n",
    "colors = ['blue', 'green', 'red', 'cyan', \n",
    "          'magenta', 'yellow', 'black', \n",
    "          'pink', 'lightgreen', 'lightblue', \n",
    "          'gray', 'indigo', 'orange']\n",
    "\n",
    "weights, params = [], []\n",
    "for c in np.arange(-4., 6.):\n",
    "    lr = LogisticRegression(penalty='l1', C=10.**c, solver='liblinear', \n",
    "                            multi_class='ovr', random_state=0)\n",
    "    lr.fit(X_train_std, y_train)\n",
    "    weights.append(lr.coef_[1])\n",
    "    params.append(10**c)\n",
    "\n",
    "weights = np.array(weights)\n",
    "\n",
    "for column, color in zip(range(weights.shape[1]), colors):\n",
    "    plt.plot(params, weights[:, column],\n",
    "             label=df_wine.columns[column + 1],\n",
    "             color=color)\n",
    "plt.axhline(0, color='black', linestyle='--', linewidth=3)\n",
    "plt.xlim([10**(-5), 10**5])\n",
    "plt.ylabel('weight coefficient')\n",
    "plt.xlabel('C')\n",
    "plt.xscale('log')\n",
    "plt.legend(loc='upper left')\n",
    "ax.legend(loc='upper center', \n",
    "          bbox_to_anchor=(1.38, 1.03),\n",
    "          ncol=1, fancybox=True)\n",
    "# plt.savefig('images/04_07.png', dpi=300, \n",
    "#             bbox_inches='tight', pad_inches=0.2)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KSOl3a0DMq2p"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8Usc-zvQMq2p"
   },
   "source": [
    "## 순차 특성 선택 알고리즘"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "id": "PKiMemwoMq2p"
   },
   "outputs": [],
   "source": [
    "from sklearn.base import clone\n",
    "from itertools import combinations\n",
    "import numpy as np\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "\n",
    "class SBS():\n",
    "    def __init__(self, estimator, k_features, scoring=accuracy_score,\n",
    "                 test_size=0.25, random_state=1):\n",
    "        self.scoring = scoring\n",
    "        self.estimator = clone(estimator)\n",
    "        self.k_features = k_features\n",
    "        self.test_size = test_size\n",
    "        self.random_state = random_state\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        \n",
    "        X_train, X_test, y_train, y_test = \\\n",
    "            train_test_split(X, y, test_size=self.test_size,\n",
    "                             random_state=self.random_state)\n",
    "\n",
    "        dim = X_train.shape[1]\n",
    "        self.indices_ = tuple(range(dim))\n",
    "        self.subsets_ = [self.indices_]\n",
    "        score = self._calc_score(X_train, y_train, \n",
    "                                 X_test, y_test, self.indices_)\n",
    "        self.scores_ = [score]\n",
    "\n",
    "        while dim > self.k_features:\n",
    "            scores = []\n",
    "            subsets = []\n",
    "\n",
    "            for p in combinations(self.indices_, r=dim - 1):\n",
    "                score = self._calc_score(X_train, y_train, \n",
    "                                         X_test, y_test, p)\n",
    "                scores.append(score)\n",
    "                subsets.append(p)\n",
    "\n",
    "            best = np.argmax(scores)\n",
    "            self.indices_ = subsets[best]\n",
    "            self.subsets_.append(self.indices_)\n",
    "            dim -= 1\n",
    "\n",
    "            self.scores_.append(scores[best])\n",
    "        self.k_score_ = self.scores_[-1]\n",
    "\n",
    "        return self\n",
    "\n",
    "    def transform(self, X):\n",
    "        return X[:, self.indices_]\n",
    "\n",
    "    def _calc_score(self, X_train, y_train, X_test, y_test, indices):\n",
    "        self.estimator.fit(X_train[:, indices], y_train)\n",
    "        y_pred = self.estimator.predict(X_test[:, indices])\n",
    "        score = self.scoring(y_test, y_pred)\n",
    "        return score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "id": "Hr7lY6fLMq2p",
    "outputId": "8a607f5d-3cc3-4918-a203-fc06fa7322c4"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxV1bn/8c9DGBJmEI0QEBwYxAk0gsq9vTjT1gGniu21Wq3UOvS219rK/bXV2kFvaW/ba61Kq3WolVoHLm2xaIU4Qk0QZJIgYhiCKJoECATI8Pz+ODv0GAIc4Oyz9zn5vl+v8+LscT2L5OQ5e6+11zJ3R0REJG7aRR2AiIhIa5SgREQklpSgREQklpSgREQklpSgREQkltpHHUC69OnTxwcNGhR1GPtky5YtdOnSJeowQpPL9cvlukFu1y+X6wbZWb958+Z95O4Ht1yfMwlq0KBBlJWVRR3GPikpKWHs2LFRhxGaXK5fLtcNcrt+uVw3yM76mdmq1tbrFp+IiMSSEpSIiMSSEpSIiMSSEpSIiMSSEpSIiMSSEpSIiMSSEpSIiMSSEpSIiMSSEpSIiMSSEpSIiMRSaEMdmdlDwHnAh+5+bCvbDfgl8BlgK3C1u78ZbLsK+E6w6w/d/ZGw4mwLps2vZPLMctbV1NGvZwG3njuU8SOLog5LUpDpn11zeZU1dRTNnZWx8jJRv1yuW3J5mapfJoQ5Ft/DwK+AR3ez/dPA4OA1GrgPGG1mvYHbgWLAgXlmNt3dq0OMNWdNm1/JpGcWUVffCEBlTR2TnlkEkPW/vLku0z+7XC4vl+sWRXmZElqCcveXzWzQHna5EHjU3R2Ya2Y9zawvMBZ4wd2rAMzsBWAc8ERYseayyTPLd/7SNqurb2TyzPKs/sVtC3b3s5v0zCJeWr4h7eX9bfH6nC0vl+u2p/Ky/XMe5WjmRcCapOW1wbrdrd+FmU0EJgIUFhZSUlISSqBhqa2tDT3mypq63a4Pu+xM1C8qUf7s6uobeXXZurSXV1fvOVteLtdtT+Vl4nMepqyebsPdpwBTAIqLiz3bhpjPxLD4/ea8yLqN23Zd3zM/9LKzcdj/VIVdt83b6un097+zvaFpl21FPQt47bYz0l7mmLtntZoUc6G8XK7bnso7tHv4n/MwRdmLrxIYkLTcP1i3u/WyH44r6tHq+t6dO7J1R0OGo5FUVNbUcdn9c9jR0ESHPPvEtoIOedx67tBQyr313KEUdMjLyfJyuW67Kw9gW30j5es3h1JmJkSZoKYDX7SEU4CN7v4+MBM4x8x6mVkv4JxgneyjNVVbmb18AycO6ElRzwIMKOqZz0Uj+rHk/U1c/sBcPti069WVRGfh2hrG3/saldV1PHrtKCZfekLSz66Auy4+LrQ2hfEji7jr4uMo6lkAGS4v7Prlct1alkdQ3i3nDKFTh3Zcct/robR7ZYS7h/Ii0anhfaCeRDvStcD1wPXBdgPuBd4FFgHFScdeA6wIXl9KpbyTTjrJs83s2bNDPf9XHi3zYd95ztfVbN1l2wtL1vvR333OT/nx331J5cZQyg+7flEKo27PLXrfh35nho+5+0Vfvn5T2s+/L/Szy17J9VtXs9XH/eJlP2LSX/3RORXRBbUXQJm38nc9tCsod7/C3fu6ewd37+/uD7r7/e5+f7Dd3f1Gdz/S3Y9z97KkYx9y96OC1+/CijGXvfrOR/xtyXpuOuMo+vYo2GX7WcMLefIrp+IOl93/OrOXfRhBlAKJL4kPvPQuX318Hkf37c6zN4xhcGG3qMOSHNC3RwFPXX8qY4cczHenLeYHf1lKY1PrHSriSCNJ5KD6xia+/+clHNa7M9f+y+G73e/Yoh5Mu3EMg/p04dpHSnnk9YrMBSlA4mf1X88u4q7nlvGZ4/ryxHWncHC3TlGHJTmkS6f2TPliMV8aM4gHX32Przw2jy3bs6P9WQkqBz02ZxXvfFjLd88bTn4rDafJDu2Rz5NfOZUzhhVy+/Ql3DF9SVZ9w8pmG+vq+dLvSnnijTXcdPpR3DNh5F5/XiL7I6+dcfv5x3Dnhccwa9kHfO6BOaxvpXdv3ChB5ZiParfz878v51NDDuasow9J6ZgundrzwJUn8eV/OZyHX6/gukfLqM2Sb1jZak3VVi6573X+8d7H/PSyE/jmuUNp1872fqDIAfjiqYN48OqTqfhoCxfe+yqLKzdGHdIeKUHlmJ/OLKduRyPfO284ieEOU5PXzvjOecP5wfhjeWn5Bi67fw7rdvOgqByYeauqGX/va2zYvJ1HrxnNpSf1jzokaUNOH3oIT331NPLMuOz+Obyw9IOoQ9otJagcsnBtDX8sW8OXxgziqEO67tc5rjxlIA9dfTJrqrYy/t7XWLQ23t+wss2f31rHFb+ZS7f89jx7w2mceuRBUYckbdDRfbsz7cYxDCnsysTHyvjtKyube1DHihJUjmhqcu6YvoSDunTia2cOPqBz/duQg3n6q6fRIa8dn3tgDjOXrE9TlG2Xu3PPi+9w8xPzGdG/J8/cMIYjDt6/LxEi6XBI93ymTjyVccccyg//+jbf/b/FNDTuOnJJlJSgcsS0BZW8ubqGb48bSrf8Dgd8vqGHduPZG09jyKHduP7385jy8rux/IaVDbY3NHLLk2/xsxeWc9HIIh778ih6d+kYdVgiFHTM497Pn8j1/3Ykv5+7mmseKWPztvqow9pJCSoH1G5v4K7nlnHCgJ5ccmL62jMO6ZbPHyeewmeO7cuPZyzjv55dTH3MvmHFXfWWHVz54Bs8M7+S/zx7CP/zuRPo1F499SQ+2rUzbvv0MO6++DheX/ERl943h7XVW6MOC1CCygn3zHqHDZu38/0Ljkl7T7D8Dnncc8VIbhh7JE+8sZov/a6UjXXx+YYVZys31HLRr19jweoafjlhBF87c/A+dVwRyaQJow7jkWtGsW5jHePvfZ0Fa2qiDkkJKtut3FDLQ6++x2Un9WfEgJ6hlNGunfGtccP4yaXHM3flx1x63+usqYrHN6y4mrvyYy6+73U2bWvgD9eN5sIR2Tsnj7QdY47qw7M3nEZBx3Zc/sAcZix6P9J4lKCy3A/+spT89nl8a9yw0Mv6XPEAHr12FB9s2sb4e19j3ipNctyap+et5coH/8FBXTry7A2nUTyod9QhiaTsqEO6Me2GMRxb1IMbHn+T+0qia39Wgspis5Z9wOzyDfzHWYMzNjzOaUf24dkbx9A1vz1X/GYuf34r/ZOvZaumJudnz5dzy5/e4uRBvXnmq2MYeFCXqMMS2WcHde3E418ezfkn9OO//7aMbz+9kB2tzE0WNiWoLLW9oZE7/7yUIw/uwhdPHZTRso88uCvP3jCG44t6cPMT8/nVrHfafA+/bfWNfG3qfO6ZtYLLiwfwyDWj6NH5wHtTikQlv0Me/xu0nT5ZtparHnqDjVsz2/6sBJWlHnq1goqPt3L7+cfQsX3mf4y9u3Tk8etGM35EP376/HK++adovmHFwUe12/n8b+byl4XvJ3pDXXIcHfL00ZLsZ2Y7e5+WrariovteY9XHWzJWvj5FWeiDTdu4Z9Y7nD28kE8NOTiyODq1z+Pnl4/g62cN5uk3E+0uNVt3RBZPFN75YDMX/fo1lqzbxH1fSDxPop56kmsuPrE/v792NFVbdjD+3tcorajKSLntM1KKpNXdzy2jocn57meHRx0KZsbXzxrCoIO68K2nFnLRr1/n86MO4+HXK6isqaNo7ixuPXdoaDOJTptfyeSZ5ayrqaNfz4JQy0our7KmjoNeeYHa7fV0y+/IH79yami9KEXiYPQRBzHthjFc83ApX/jNP7j85P7MWrYh1M+erqCyTFlFFc/Or2Tivx7BYQd1jjqcncaPLOLx60bzwcY6fjTjbSqDgWYra+qY9Mwips2vTHuZ0+ZXMumZRVTW1OEhl9WyPICPt+xgR4Nzw9gjlJykTRjUpwvP3HAaA3oX8Njc1aF/9nQFlUUam5zbpy/h0O753HD6kVGHs4uTB/WmW0EHttZv/8T6uvpG/vPJBfzwr2+ntbyqLdtpOXVVWGXtrjwHHny1gmv+5Yi0lycSRz07d6RuR+Mu6+vqG5k8szytV1FKUFnkj6VrWLJuE/97xUg6d4znj+7DTdtbXd/kcM4xhWkt6w//WJ2xsvZUnqYlkbbm/d1Mdpjuz0I8/8rJLjZurWfyzGWMGtSb84/vG3U4u9WvZ8HOW2DJinoW8OOLjktrWS+Vb8hYWXsqr1/PgrSXJRJnu/ucp/uzoDaoLPHzvy9nY109t1+wbxMRZtqt5w6loMW05QUd8rj13KFZXVYU5YnEVaY+C7qCygLL1m/isbmr+PzowzimX4+ow9mj5vvPzT3dikLsWZdcViZ68WWybiJxlqnPnhJUzLk735++lG757bnl7Oz4pj5+ZBHjRxZRUlLC2LFjM1JWpmSybiJxlonPnm7xxdxzi9czZ+XH3HLOUHppkjsRaUOUoGKsbkcjP/rr2xzdtzufH3VY1OGIiGSUElSM3f/Su1TW1HHH+cPJS/NEhCIicacEFVNrqrZy/0vvcv4J/Rh9xEFRhyMiknFKUDH14xlv086MSZ8OfyJCEZE4CjVBmdk4Mys3sxVmdlsr2wea2YtmttDMSsysf9K2RjNbELymhxln3Ly24iOeW7yeG08/Ug+BikibFVo3czPLA+4FzgbWAqVmNt3dlybt9lPgUXd/xMzOAO4Crgy21bn7iLDii6v6xia+/+clDOhdwJf/VeO7iUjbFeYV1ChghbuvdPcdwFTgwhb7DAdmBe9nt7K9zfn93FUs/6CW7352OPktntQWEWlLLKypus3sUmCcu385WL4SGO3uNyXt8wfgH+7+SzO7GHga6OPuH5tZA7AAaADudvdprZQxEZgIUFhYeNLUqVNDqUtYamtr6dq1687lTTucb7+8lSN75HFLcadYD2mUipb1yyW5XDfI7frlct0gO+t3+umnz3P34pbrox5J4pvAr8zsauBloBJoHsd9oLtXmtkRwCwzW+Tu7yYf7O5TgCkAxcXFnm1P9rccjWDSMwupb6rjF1eN4ahDukUXWJrk8mgLuVw3yO365XLdILfqF2aCqgQGJC33D9bt5O7rgIsBzKwrcIm71wTbKoN/V5pZCTAS+ESCyiWL1m5kaukarh1zeE4kJxGRAxVmG1QpMNjMDjezjsAE4BO98cysj5k1xzAJeChY38vMOjXvA4wBkjtX5BR35/bpizmoS0e+dtbgqMMREYmF0BKUuzcANwEzgbeBJ919iZndaWYXBLuNBcrNbDlQCPwoWH80UGZmb5HoPHF3i95/OeXZ+ZW8ubqGb40bRvf8DlGHIyISC6G2Qbn7DGBGi3XfS3r/FPBUK8e9DqR/xrkYqt3ewF3PLeOE/j249MT+ez9ARKSNiLqTRJt3z6x32LB5O1OuPIl2Gm9PRGQnJagIrd/SxEOvv8elJ/Vn5GG9og5HRCRWlKAiMG1+5c5ZWQ04vn/3qEMSEYkdJagMmza/kknPLKKuPvG4lwN3zSine35HTR0uIpJEo5ln2OSZ5TuTU7O6+kYmzyyPKCIRkXhSgsqwdTV1+7ReRKStUoLKsN1Nn6FpNUREPkkJKsNuPXco7Vt0Jy/okMet5w6NKCIRkXhSgsqw8SOLOLxPZzrkJZJUUc8C7rr4OHWQEBFpQb34Mqy+sYm11dv4wuiBjO2+IWdGHRYRSTddQWXY0nWbqKtv5ORBvaMORUQk1pSgMqy0ogqA4kEaOUJEZE+UoDKsrKKaw3p3prB7ftShiIjEmhJUBrk7ZauqdPUkIpICJagMqvh4Kx/V7lD7k4hICpSgMqi5/elkXUGJiOyVElQGlVVU0atzB448uGvUoYiIxJ4SVAaVVlRz0sDemGliQhGRvVGCypANm7fz3kdbdHtPRCRFSlAZMm9V8/NP6iAhIpIKJagMKa2oplP7dhxbpNlzRURSoQSVIWUVVZwwoCed2udFHYqISFZQgsqArTsaWLxuk9qfRET2gRJUBixYXUNjk6v9SURkHyhBZUBpRTVmcOJhuoISEUmVElQGlK2qYmhhN3oUdIg6FBGRrKEEFbKGxibeXFWt8fdERPZRqAnKzMaZWbmZrTCz21rZPtDMXjSzhWZWYmb9k7ZdZWbvBK+rwowzTMvWb2bLjkaNYC4iso9CS1BmlgfcC3waGA5cYWbDW+z2U+BRdz8euBO4Kzi2N3A7MBoYBdxuZln5F/6fA8TqCkpEZF+EeQU1Cljh7ivdfQcwFbiwxT7DgVnB+9lJ288FXnD3KnevBl4AxoUYa2jKKqop6llAv54FUYciIpJVwkxQRcCapOW1wbpkbwEXB+8vArqZ2UEpHht77k5phSYoFBHZH+0jLv+bwK/M7GrgZaASaEz1YDObCEwEKCwspKSkJIQQ99+HW5v4cPN2euz4qNXYamtrYxdzOuVy/XK5bpDb9cvlukFu1S/MBFUJDEha7h+s28nd1xFcQZlZV+ASd68xs0pgbItjS1oW4O5TgCkAxcXFPnbs2Ja7ROrpeWuBt/j8OaMZduiuY/CVlJQQt5jTKZfrl8t1g9yuXy7XDXKrfmHe4isFBpvZ4WbWEZgATE/ewcz6mFlzDJOAh4L3M4FzzKxX0DninGBdVilbVUW3/PYMOaRb1KGIiGSd0BKUuzcAN5FILG8DT7r7EjO708wuCHYbC5Sb2XKgEPhRcGwV8AMSSa4UuDNYl1VKK6opHtiLdu00QaGIyL4KtQ3K3WcAM1qs+17S+6eAp3Zz7EP884oq61Rt2cGKD2u5aGTW9e0QEYkFjSQRknmrqgE9/yQisr+UoEJSVlFFx7x2HN+/R9ShiIhkJSWokJRWVHFc/x7kd9AEhSIi+0MJKgTb6htZVLlRD+iKiByAvSYoMzs/qSu4pOCtNTXUNzonD1T7k4jI/kol8VwOvGNmPzGzYWEHlAvKgg4SJw3UFZSIyP7aa4Jy938HRgLvAg+b2Rwzm2hmevp0N0orqhhS2JVeXTpGHYqISNZK6dadu28i8bzSVKAviYFd3zSzm0OMLSs1NjnzVlVTrO7lIiIHJJU2qAvM7FkSY+F1AEa5+6eBE4Bbwg0v+yz/YDObtzVwsjpIiIgckFRGkrgE+Lm7v5y80t23mtm14YSVvcqCCQqL1UFCROSApJKg7gDeb14wswKg0N0r3P3FsALLVqUV1RzaPZ/+vTRBoYjIgUilDepPQFPScmOwTlpRFkxQaKYBYkVEDkQqCap9MGU7AMF7dU9rRWVNHes2btP4eyIiaZBKgtqQND0GZnYh8FF4IWWvne1P6iAhInLAUmmDuh543Mx+BRiwBvhiqFFlqdKKKrp2at/q7LkiIrJv9pqg3P1d4JRgSnbcvTb0qLJUWUU1Jw7sRZ4mKBQROWApTVhoZp8FjgHymxv/3f3OEOPKOhu31lP+wWY+e1zfqEMREckJqTyoez+J8fhuJnGL7zJgYMhxZZ03V1fjjkaQEBFJk1Q6SZzm7l8Eqt39+8CpwJBww8o+pRVVtG9njBjQM+pQRERyQioJalvw71Yz6wfUkxiPT5KUVVRzbFEPCjpqgkIRkXRIJUH92cx6ApOBN4EK4A9hBpVttjc0smBtjcbfExFJoz12kggmKnzR3WuAp83sL0C+u2/MSHRZYnHlRnY0NKn9SUQkjfZ4BeXuTcC9ScvblZx2VVqRmKCwWBMUioikTSq3+F40s0tMg8vtVllFFUcc3IWDunaKOhQRkZyRSoL6ConBYbeb2SYz22xmm0KOK2s0NTllq6o5WdNriIikVSojSWhq9z14d0MtNVvrNf6eiEia7TVBmdmnWlvfcgLDtqq5/UkjmIuIpFcqQx3dmvQ+HxgFzAPOCCWiLFNWUUWfrp0YeFDnqEMREckpe22Dcvfzk15nA8cC1amc3MzGmVm5ma0ws9ta2X6Ymc02s/lmttDMPhOsH2RmdWa2IHjdv68Vy5TSVVWcrAkKRUTSLqXBYltYCxy9t53MLI9EF/Wzg2NKzWy6uy9N2u07wJPufp+ZDQdmAIOCbe+6+4j9iC9j1m/cxpqqOq4+7fCoQxERyTmptEHdA3iw2A4YQWJEib0ZBaxw95XBeaYCFwLJCcqB5smTegDrUgs7HspWJSYo1AgSIiLpZ+6+5x3MrkpabAAq3P21vZ7Y7FJgnLt/OVi+Ehjt7jcl7dMXeB7oBXQBznL3eWY2CFgCLAc2Ad9x91daKWMiMBGgsLDwpKlTp+4trLT6/dLtvFLZwK/P7Lxfc0DV1tbStWvXECKLh1yuXy7XDXK7frlcN8jO+p1++unz3L245fpUbvE9BWxz90ZI3Lozs87uvjUNcV0BPOzuPzOzU4HHzOxY4H3gMHf/2MxOAqaZ2THu/onnr9x9CjAFoLi42MeOHZuGkFI3eeErFB/egTPPOGW/ji8pKSHTMWdSLtcvl+sGuV2/XK4b5Fb9UhpJAihIWi4A/p7CcZXAgKTl/sG6ZNcCTwK4+xwSvQT7BEMqfRysnwe8S8ym+Ni8rZ63399EsR7QFREJRSoJKj95mvfgfSp9qkuBwWZ2uJl1BCYA01vssxo4E8DMjiaRoDaY2cFBJwvM7AhgMLAyhTIzZv7qGppczz+JiIQllQS1xcxObF4IbrnV7e0gd28AbgJmAm+T6K23xMzuNLMLgt1uAa4zs7eAJ4CrPdEo9ilgoZktIHGL8Xp3r9qXioWtrKKKvHbGiMM0QaGISBhSaYP6OvAnM1tHYsr3Q0lMAb9X7j6DRNfx5HXfS3q/FBjTynFPA0+nUkZUSiuqGd63O1077U9PfRER2ZtUxuIrNbNhwNBgVbm714cbVrzVNzYxf001V4w6LOpQRERy1l5v8ZnZjUAXd1/s7ouBrmZ2Q/ihxdeSdZvYVt+k9icRkRCl0gZ1XTCjLgDuXg1cF15I8VdWkWgO0wSFIiLhSSVB5SVPVhj0rusYXkjxV1pRxcCDOnNI9/yoQxERyVmpJKi/AX80szPN7EwSve2eCzes+HJ3yiqq9fyTiEjIUumC9m0SwwldHywvJNGTr01676MtfLxlh8bfExEJWSrTbTQB/wAqSAwAewaJ55rapLJggsJidZAQEQnVbq+gzGwIibHyrgA+Av4I4O6nZya0eCqtqKJX5w4ceXCXqEMREclpe7rFtwx4BTjP3VcAmNk3MhJVjJWtqqZ4UG9NUCgiErI93eK7mMSo4rPN7DdBB4k2/Vd5w+btvPfRFrU/iYhkwG4TlLtPc/cJwDBgNokhjw4xs/vM7JxMBRgn84IJCtX+JCISvlQ6SWxx9z+4+/kkpsyYT6JnX5tTWlFNp/btOLZfj6hDERHJeak8B7WTu1e7+xR3PzOsgOKsrKKKEQN60rH9Pv23iYjIftBf2hRt3dHA4nWbNP6eiEiGKEGlaMHqGhqbnGJ1kBARyQglqBSVVlRjBidqgFgRkYxQgkpR2aoqhh3ane75HaIORUSkTVCCSkFDYxNvrqrW808iIhmkBJWCZes3s2VHo55/EhHJICWoFJQGExTqCkpEJHOUoFJQVlFNUc8C+vYoiDoUEZE2QwlqL9yd0ooqXT2JiGSYEtRerKmq48PN29X+JCKSYUpQe/HP9iclKBGRTFKC2ouyVVV0z2/P4EO6Rh2KiEibogS1F6UViQkK27Vr01NhiYhknBLUHlRt2cGKD2s1/p6ISASUoPZg3qpqQO1PIiJRCDVBmdk4Mys3sxVmdlsr2w8zs9lmNt/MFprZZ5K2TQqOKzezc8OMc3fKKqromNeO44o0QaGISKa1D+vEZpYH3AucDawFSs1sursvTdrtO8CT7n6fmQ0HZgCDgvcTgGOAfsDfzWyIuzeGFW9rSiuqOL5/D/I75GWyWBERIdwrqFHACndf6e47gKnAhS32caB78L4HsC54fyEw1d23u/t7wIrgfBmzrb6RRZUb9fyTiEhEQruCAoqANUnLa4HRLfa5A3jezG4GugBnJR07t8WxRS0LMLOJwESAwsJCSkpK0hE3AOVVjdQ3Ovmb11JSsj5t501WW1ub1pjjJpfrl8t1g9yuXy7XDXKrfmEmqFRcATzs7j8zs1OBx8zs2FQPdvcpwBSA4uJiHzt2bNoCWzJ7BVDO1ed9ip6dO6btvMlKSkpIZ8xxk8v1y+W6QW7XL5frBrlVvzATVCUwIGm5f7Au2bXAOAB3n2Nm+UCfFI8NVWlFFUMKu4aWnEREZM/CbIMqBQab2eFm1pFEp4fpLfZZDZwJYGZHA/nAhmC/CWbWycwOBwYDb4QY6yc0NjnzVlWr/UlEJEKhXUG5e4OZ3QTMBPKAh9x9iZndCZS5+3TgFuA3ZvYNEh0mrnZ3B5aY2ZPAUqABuDGTPfiWf7CZzdsaNIK5iEiEQm2DcvcZJLqOJ6/7XtL7pcCY3Rz7I+BHYca3O2XBALHFA3UFJSISFY0k0YrSimoO7Z5P/16aoFBEJCpKUC00T1BYPKgXZhogVkQkKkpQLVTW1PH+xm0af09EJGJKUC2UVSQGiNUI5iIi0VKCaqG0ooqundoz7NDue99ZRERCowTVQllFNScO7EWeJigUEYmUElSSjVvrKf9gMycP1O09EZGoKUElmbc6eP5JHSRERCKnBJWktKKa9u2MEQN6Rh2KiEibpwSVpKyiimOLelDQURMUiohETQkKmDa/ktPuepHSimpWfLiZafMzOnC6iIi0Iur5oCI3bX4lk55ZRF19Yiza2u2NTHpmEQDjR+4yR6KIiGRIm7+CmjyzfGdyalZX38jkmeURRSQiIqAExbqaun1aLyIimdHmE1S/nq2PWL679SIikhltPkHdeu5QCjp8stdeQYc8bj13aEQRiYgIqJPEzo4Qk2eWs66mjn49C7j13KHqICEiErE2n6AgkaSUkERE4qXN3+ITEZF4UoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYCjVBmdk4Mys3sxVmdlsr239uZguC13Izq0na1pi0bXqYcYqISPyENhafmeUB9wJnA2uBUjOb7u5Lm/dx928k7X8zMDLpFHXuPiKs+EREJN7CvIIaBaxw95Xuvss+ZVIAAApbSURBVAOYCly4h/2vAJ4IMR4REcki5u7hnNjsUmCcu385WL4SGO3uN7Wy70BgLtDf3RuDdQ3AAqABuNvdp7Vy3ERgIkBhYeFJU6dODaUuYamtraVr165RhxGaXK5fLtcNcrt+uVw3yM76nX766fPcvbjl+rhMtzEBeKo5OQUGunulmR0BzDKzRe7+bvJB7j4FmAJQXFzsY8eOzVjA6VBSUkK2xbwvcrl+uVw3yO365XLdILfqF+YtvkpgQNJy/2BdaybQ4vaeu1cG/64ESvhk+5SIiOS4MBNUKTDYzA43s44kktAuvfHMbBjQC5iTtK6XmXUK3vcBxgBLWx4rIiK5K7RbfO7eYGY3ATOBPOAhd19iZncCZe7enKwmAFP9k41hRwMPmFkTiSR6d3LvPxERyX2htkG5+wxgRot132uxfEcrx70OHBdmbCIiEm8aSUJERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGJJCUpERGIp1ARlZuPMrNzMVpjZba1s/7mZLQhey82sJmnbVWb2TvC6Ksw4RUQkftqHdWIzywPuBc4G1gKlZjbd3Zc27+Pu30ja/2ZgZPC+N3A7UAw4MC84tjqseEVEJF7CvIIaBaxw95XuvgOYCly4h/2vAJ4I3p8LvODuVUFSegEYF2KsIiISM6FdQQFFwJqk5bXA6NZ2NLOBwOHArD0cW9TKcROBicFirZmVH2DMmdYH+CjqIEKUy/XL5bpBbtcvl+sG2Vm/ga2tDDNB7YsJwFPu3rgvB7n7FGBKOCGFz8zK3L046jjCksv1y+W6QW7XL5frBrlVvzBv8VUCA5KW+wfrWjOBf97e29djRUQkB4WZoEqBwWZ2uJl1JJGEprfcycyGAb2AOUmrZwLnmFkvM+sFnBOsExGRNiK0W3zu3mBmN5FILHnAQ+6+xMzuBMrcvTlZTQCmursnHVtlZj8gkeQA7nT3qrBijVDW3p5MUS7XL5frBrldv1yuG+RQ/SwpL4iIiMSGRpIQEZFYUoISEZFYUoKKgJkNMLPZZrbUzJaY2X9EHVO6mVmemc03s79EHUu6mVlPM3vKzJaZ2dtmdmrUMaWLmX0j+J1cbGZPmFl+1DEdCDN7yMw+NLPFSet6m9kLwTBqLwQdsbLSbuo3OfjdXGhmz5pZzyhjPBBKUNFoAG5x9+HAKcCNZjY84pjS7T+At6MOIiS/BP7m7sOAE8iReppZEfA1oNjdjyXRuWlCtFEdsIfZdRSa24AX3X0w8GKwnK0eZtf6vQAc6+7HA8uBSZkOKl2UoCLg7u+7+5vB+80k/sDtMlJGtjKz/sBngd9GHUu6mVkP4FPAgwDuvsPda/Z8VFZpDxSYWXugM7Au4ngOiLu/DLTsAXwh8Ejw/hFgfEaDSqPW6ufuz7t7Q7A4l8RzpFlJCSpiZjaIxCC5/4g2krT6BfAtoCnqQEJwOLAB+F1wC/O3ZtYl6qDSwd0rgZ8Cq4H3gY3u/ny0UYWi0N3fD96vBwqjDCZk1wDPRR3E/lKCipCZdQWeBr7u7puijicdzOw84EN3nxd1LCFpD5wI3OfuI4EtZPctop2CtpgLSSThfkAXM/v3aKMKV/D8ZU4+a2Nm/49Ec8LjUceyv5SgImJmHUgkp8fd/Zmo40mjMcAFZlZBYgT7M8zs99GGlFZrgbXu3nzF+xSJhJULzgLec/cN7l4PPAOcFnFMYfjAzPoCBP9+GHE8aWdmVwPnAV/wLH7YVQkqAmZmJNow3nb3/4k6nnRy90nu3t/dB5FoYJ/l7jnzLdzd1wNrzGxosOpMYOkeDskmq4FTzKxz8Dt6JjnSAaSF6UDzJKhXAf8XYSxpZ2bjSNxiv8Ddt0Ydz4FQgorGGOBKElcXzTMKfybqoCRlNwOPm9lCYATw44jjSYvgqvAp4E1gEYm/D1k9bI6ZPUFinM+hZrbWzK4F7gbONrN3SFw13h1ljAdiN/X7FdANeCH423J/pEEeAA11JCIisaQrKBERiSUlKBERiSUlKBERiSUlKBERiSUlKBERiSUlKGkTzMzN7GdJy980szvSdO6HzezSdJxrL+VcFoyePruVbZODUcgn78d5R+gxB4kjJShpK7YDF5tZn6gDSRYMypqqa4Hr3P30VrZNBI5391v3I4wRwD4lKEvQ3w8JlX7BpK1oIPHQ6Tdabmh5BWRmtcG/Y83sJTP7PzNbaWZ3m9kXzOwNM1tkZkcmneYsMyszs+XBeITNc2JNNrPSYG6erySd9xUzm04ro1CY2RXB+Reb2X8H674H/AvwYMurpOA8XYF5Zna5mR1sZk8H5Zaa2Zhgv1FmNicY5PZ1MxtqZh2BO4HLg4c6LzezO8zsm0nnX2xmg4JXuZk9CiwGBpjZrUn1+36wfxcz+6uZvRUce/m+/rBEIDHwpUhbcS+w0Mx+sg/HnAAcTWJKg5XAb919lCUmmbwZ+Hqw3yBgFHAkMNvMjgK+SGJE8JPNrBPwmpk1jw5+Iok5e95LLszM+gH/DZwEVAPPm9l4d7/TzM4AvunuZcnHuPsFZlbr7iOCc/wB+Lm7v2pmhwEzgzosA/7V3RvM7Czgx+5+SZD8it39puD4O/bw/zEYuMrd55rZOcHyKMCA6Wb2KeBgYJ27fzY4X48U/p9FdqEEJW2Gu28Kvv1/DahL8bDS5qkZzOxdoDnBLAKSb7U96e5NwDtmthIYBpwDHJ90ddaDxB/0HcAbLZNT4GSgxN03BGU+TmL+qWkpxguJ4XuGJ4bTA6C7JUbO7wE8YmaDSYzg3WEfztlslbvPDd6fE7zmB8tdSdTvFeBnwdXfX9z9lf0oR0QJStqcX5AYa+53SesaCG53B+0qHZO2bU9635S03MQnPz8txwxzElcVN7v7zOQNZjaWxDQdYWkHnOLu21qU+ytgtrtfZIl5yEp2c/zO/49A8rTvyXEbcJe7P9DyBGZ2Iol2rR+a2Yvufue+VkJEbVDSprh7FfAkiQ4HzSpI3FIDuID9u7K4zMzaBe1SRwDlJG6tfdUSU6tgZkNs75MbvgH8m5n1MbM84ArgpX2M5XkStx8Jyh0RvO0BVAbvr07afzOJwUWbVRBMIRIkmsN3U85M4Jrg6gwzKzKzQ4LblFvd/ffAZHJnOhLJMCUoaYt+BiT35vsNiaTwFnAq+3d1s5pEcnkOuD64evktiU4Qb5rZYuAB9nLXIrideBswG3gLmOfu+zodxNeA4qDjwlLg+mD9T4C7zGx+izhmk7gluCDo0PA00NvMlgA3Act3E+vzwB+AOWa2iMRI6N2A44A3zGwBcDvww32MXwTQaOYiIhJTuoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFYUoISEZFY+v+EaFTZA/EAJAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "knn = KNeighborsClassifier(n_neighbors=5)\n",
    "\n",
    "# 특성을 선택합니다\n",
    "sbs = SBS(knn, k_features=1)\n",
    "sbs.fit(X_train_std, y_train)\n",
    "\n",
    "# 선택한 특성의 성능을 출력합니다\n",
    "k_feat = [len(k) for k in sbs.subsets_]\n",
    "\n",
    "plt.plot(k_feat, sbs.scores_, marker='o')\n",
    "plt.ylim([0.7, 1.02])\n",
    "plt.ylabel('Accuracy')\n",
    "plt.xlabel('Number of features')\n",
    "plt.grid()\n",
    "plt.tight_layout()\n",
    "# plt.savefig('images/04_08.png', dpi=300)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HYOAOJwLMq2p",
    "outputId": "087d471f-579a-481f-b3ce-d79ecf78430d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Alcohol', 'Malic acid', 'OD280/OD315 of diluted wines'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "k3 = list(sbs.subsets_[10])\n",
    "print(df_wine.columns[1:][k3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zP-Au4A8Mq2p",
    "outputId": "feaff4e3-4ea9-4fab-8ef0-ccea038731a5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Alcohol', 'Malic acid', 'OD280/OD315 of diluted wines'], dtype='object')"
      ]
     },
     "execution_count": 80,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_wine.columns[1:][k3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Z2lUfmEgMq2p",
    "outputId": "4a09f4c2-37b4-4efc-c160-89018ccc07fb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "훈련 정확도: 0.967741935483871\n",
      "테스트 정확도: 0.9629629629629629\n"
     ]
    }
   ],
   "source": [
    "knn.fit(X_train_std, y_train)\n",
    "print('훈련 정확도:', knn.score(X_train_std, y_train))\n",
    "print('테스트 정확도:', knn.score(X_test_std, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ygpbt2QKMq2q",
    "outputId": "61ac503b-4df2-4827-83f4-cc6c43a51afe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "훈련 정확도: 0.9516129032258065\n",
      "테스트 정확도: 0.9259259259259259\n"
     ]
    }
   ],
   "source": [
    "knn.fit(X_train_std[:, k3], y_train)\n",
    "print('훈련 정확도:', knn.score(X_train_std[:, k3], y_train))\n",
    "print('테스트 정확도:', knn.score(X_test_std[:, k3], y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bcPrtTYPMq2q"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LgtCOK-OMq2q"
   },
   "source": [
    "# 랜덤 포레스트의 특성 중요도 사용"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 522
    },
    "id": "DSbLfXAgMq2q",
    "outputId": "81dd66c9-424a-4628-a14e-b0de1380fa35"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 1) Proline                        0.185453\n",
      " 2) Flavanoids                     0.174751\n",
      " 3) Color intensity                0.143920\n",
      " 4) OD280/OD315 of diluted wines   0.136162\n",
      " 5) Alcohol                        0.118529\n",
      " 6) Hue                            0.058739\n",
      " 7) Total phenols                  0.050872\n",
      " 8) Magnesium                      0.031357\n",
      " 9) Malic acid                     0.025648\n",
      "10) Proanthocyanins                0.025570\n",
      "11) Alcalinity of ash              0.022366\n",
      "12) Nonflavanoid phenols           0.013354\n",
      "13) Ash                            0.013279\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAAEYCAYAAAAeWvJ8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd7hcVdn+8e9N6CUUQVBaEAhFlBZABRFsgA2kCQiCYgEUUVRefEWavhaw/rCBFGkCoRqVLl1ASOhB0BBaKEqTXky4f3+sNZzJMDmZk7PLmZnnc13nyszeM/tZc7LPrL1XeZZsE0IIIdRlrroLEEIIob9FRRRCCKFWURGFEEKoVVREIYQQahUVUQghhFpFRRRCCKFWURGFEEKoVVREYUSSdJ+kFyU91/Tz5gKO+f6iythBvEMlnVJVvMFI2kPSNXWXI4R2oiIKI9lHbS/c9PNwnYWRNHed8edUt5Y79I+oiEJXkbSopOMkPSLpIUnflTQq71tZ0mWSnpD0uKRTJS2W950MrAD8Md9dHSBpM0nTWo7/2l1TvqM5S9Ipkp4B9hgsfgdlt6R9JP1T0rOSvpPLfK2kZySNlzRvfu1mkqZJ+t/8We6T9MmW38NJkh6TdL+kgyTNlfftIemvkn4q6QngDOA3wDvzZ/9Pft2HJd2cYz8o6dCm44/J5d1d0gO5DN9q2j8ql+2e/FkmSVo+71td0iWSnpR0t6Qdh/jfHPpMVESh2/wOmA6sAqwLfBD4bN4n4PvAm4E1gOWBQwFs7wY8wMBd1hEdxtsaOAtYDDh1NvE7sQWwPvAO4ADgGGDXXNa1gJ2bXrsMsCSwLLA7cIyk1fK+o4BFgbcA7wE+BXy66b0bAVOBpfPx9wKuy599sfya5/P7FgM+DOwtaZuW8m4CrAa8DzhY0hp5+/65rB8CRgOfAV6QtBBwCfB74I3ATsCvJK05hN9R6DNREYWR7DxJ/8k/50lamvTF9xXbz9v+N/BT0pcdtqfYvsT2y7YfA35C+pIejutsn2f7VdIX7izjd+gI28/YngzcAVxse6rtp4ELSJVbs2/nz3Ml8Gdgx3wHthPwTdvP2r4P+DGwW9P7HrZ9lO3ptl9sVxDbV9i+3fartm8DTuP1v6/DbL9o+1bgVmDtvP2zwEG273Zyq+0ngI8A99k+Ice+GTgb2GEIv6PQZ6LtOIxk29i+tPFE0obAPMAjkhqb5wIezPuXBn4OvBtYJO97aphleLDp8YqDxe/Qv5oev9jm+TJNz5+y/XzT8/tJd3tL5nLc37Jv2VmUuy1JGwE/IN2JzQvMB5zZ8rJHmx6/ACycHy8P3NPmsCsCGzWa/7K5gZNnV57Qv+KOKHSTB4GXgSVtL5Z/Rtt+a97/PcDA22yPJjVJqen9ranmnwcWbDzJdxpLtbym+T2zi1+0xXNTV8MKwMPA48B/SV/6zfsemkW52z2H1Hw2AVje9qKkfiS1eV07DwIrz2L7lU2/n8Vyc+DeHR439KGoiELXsP0IcDHwY0mjJc2VO/sbzUmLAM8BT0taFvhGyyH+RepTafgHMH/utJ8HOIh0VzCn8ctwmKR5Jb2b1Ox1pu0ZwHjg/yQtImlFUp/NYEPF/wUs1xgMkS0CPGn7pXy3ucsQynUs8B1Jqyp5u6Q3AH8CxkraTdI8+WeDpr6lEF4nKqLQbT5Faka6k9TsdhbwprzvMGA94GlSf8o5Le/9PnBQ7nP6eu6X2Yf0pfoQ6Q5pGoMbLH7RHs0xHiYNlNjL9l15376k8k4FriHd3Rw/yLEuAyYDj0p6PG/bBzhc0rPAwaTKrVM/ya+/GHgGOA5YwPazpAEcO+VyPwr8kEEq+BAUC+OFMPJI2gw4xfZydZclhLLFHVEIIYRaRUUUQgihVtE0F0IIoVZxRxRCCKFWI25C65JLLukxY8bUXYwQQggFmjRp0uO2W+fpASOwIhozZgwTJ06suxghhBAKJOn+We2LprkQQgi1iooohBBCraIiCiGEUKsR10dUhDEH/rnwY973gw8XfswQQghxRxRCCKFmURGFEEKoVVREIYQQatWTfURVKaMvCqI/KoTQX+KOKIQQQq2iIgohhFCrqIhCCCHUKiqiEEIItYqKKIQQQq2iIgohhFCrqIhCCCHUKiqiEEIIteqoIpK0paS7JU2RdGCb/ZtKuknSdEnbt+ybIemW/DOhqIKHEELoDbPNrCBpFPBL4APANOBGSRNs39n0sgeAPYCvtznEi7bXKaCsIYQQelAnKX42BKbYngog6XRga+C1isj2fXnfqyWUMYQQQg/rpGluWeDBpufT8rZOzS9poqTrJW3T7gWSPp9fM/Gxxx4bwqFDCCF0uyoGK6xoexywC/AzSSu3vsD2MbbH2R631FJLVVCkEEIII0UnTXMPAcs3PV8ub+uI7Yfyv1MlXQGsC9wzhDIGYtXZEELv6uSO6EZgVUkrSZoX2AnoaPSbpMUlzZcfLwlsTFPfUgghhDDbisj2dOBLwEXA34HxtidLOlzSxwAkbSBpGrADcLSkyfntawATJd0KXA78oGW0XQghhD7X0cJ4ts8Hzm/ZdnDT4xtJTXat77sWeNswyxhCCKGHxQqt4XWiPyqEUKVI8RNCCKFWURGFEEKoVVREIYQQahUVUQghhFpFRRRCCKFWURGFEEKoVVREIYQQahXziEKtYs5SCCHuiEIIIdQqKqIQQgi1iooohBBCraIiCiGEUKuOKiJJW0q6W9IUSQe22b+ppJskTZe0fcu+3SX9M//sXlTBQwgh9IbZVkSSRgG/BLYC1gR2lrRmy8seAPYAft/y3iWAQ4CNgA2BQyQtPvxihxBC6BWd3BFtCEyxPdX2K8DpwNbNL7B9n+3bgFdb3rsFcIntJ20/BVwCbFlAuUMIIfSITiqiZYEHm55Py9s60dF7JX1e0kRJEx977LEODx1CCKEXjIjBCraPsT3O9rilllqq7uKEEEKoUCcV0UPA8k3Pl8vbOjGc94YQQugDnVRENwKrSlpJ0rzATsCEDo9/EfBBSYvnQQofzNtCCCEEoIOKyPZ04EukCuTvwHjbkyUdLuljAJI2kDQN2AE4WtLk/N4nge+QKrMbgcPzthBCCAHoMOmp7fOB81u2Hdz0+EZSs1u79x4PHD+MMoYQQuhhI2KwQgghhP4VFVEIIYRaRUUUQgihVlERhRBCqFVURCGEEGoVFVEIIYRaRUUUQgihVlERhRBCqFVURCGEEGoVFVEIIYRaRUUUQgihVlERhRBCqFVURCGEEGrVUfbtELrdmAP/XMpx7/vBh0s5bgj9pKM7IklbSrpb0hRJB7bZP5+kM/L+v0kak7ePkfSipFvyz2+KLX4IIYRuN9s7IkmjgF8CHwCmATdKmmD7zqaX7Qk8ZXsVSTsBPwQ+kffdY3udgssdQgihR3RyR7QhMMX2VNuvAKcDW7e8ZmvgxPz4LOB9klRcMUMIIfSqTvqIlgUebHo+DdhoVq+xPV3S08Ab8r6VJN0MPAMcZPvq1gCSPg98HmCFFVYY0gcIYaSJ/qgQhqbsUXOPACvYXhfYH/i9pNGtL7J9jO1xtscttdRSJRcphBDCSNJJRfQQsHzT8+XytravkTQ3sCjwhO2XbT8BYHsScA8wdriFDiGE0Ds6qYhuBFaVtJKkeYGdgAktr5kA7J4fbw9cZtuSlsqDHZD0FmBVYGoxRQ8hhNALZttHlPt8vgRcBIwCjrc9WdLhwETbE4DjgJMlTQGeJFVWAJsCh0v6L/AqsJftJ8v4ICGEELpTRxNabZ8PnN+y7eCmxy8BO7R539nA2cMsYwghhB4WKX5CCCHUKiqiEEIItYqKKIQQQq0i6WkIXayMybMxcTZULe6IQggh1CruiEIIHanq7itSJPWfuCMKIYRQq7gjCiH0rbj7GhnijiiEEEKtoiIKIYRQq6iIQggh1Cr6iEIIoQJVzvnqtvllcUcUQgihVlERhRBCqFVHFZGkLSXdLWmKpAPb7J9P0hl5/98kjWna9828/W5JWxRX9BBCCL1gthVRXmH1l8BWwJrAzpLWbHnZnsBTtlcBfgr8ML93TdIieW8FtgR+1VixNYQQQoDO7og2BKbYnmr7FeB0YOuW12wNnJgfnwW8T5Ly9tNtv2z7XmBKPl4IIYQAgGwP/gJpe2BL25/Nz3cDNrL9pabX3JFfMy0/vwfYCDgUuN72KXn7ccAFts9qifF54PP56WrA3cP/aB1ZEng8YnVFrF78TL0aqxc/U6/GqvIzrWh7qXY7RsTwbdvHAMdUHVfSRNvjItbIj9WLn6lXY/XiZ+rVWFV+psF00jT3ELB80/Pl8ra2r5E0N7Ao8ESH7w0hhNDHOqmIbgRWlbSSpHlJgw8mtLxmArB7frw9cJlTm98EYKc8qm4lYFXghmKKHkIIoRfMtmnO9nRJXwIuAkYBx9ueLOlwYKLtCcBxwMmSpgBPkior8uvGA3cC04Ev2p5R0meZE1U2B0as7ogTsbonTsTqnjiDmu1ghRBCCKFMkVkhhBBCraIiCiGEUKuoiHqApDfUXYYQQphTURFVRNJckkaXdPjrJZ0p6UM5o0VpJC0kaa78eKykj0map8yYvUTSEZJGS5pH0l8kPSZp17rLNSckLTHYT93lGy5JG0u6RNI/JE2VdK+kqSXF6pnzYk705WAFSZsAq9o+QdJSwMI5BVHRcX4P7AXMIA2DHw383PaRBccR8H7gM8AGwHjgd7b/UWScHGsS8G5gceCvpM/1iu1PFnT8o4BZnpS2v1xEnFnELv28kHSL7XUkfRz4CLA/cJXttYuM0xTvI8B3gBVJo2QF2PawL4ok3Uv6vxKwAvBUfrwY8IDtlYYbYxZxFwJetP2qpLHA6qSMLf8tOM5dwFeBSaS/YQBsP1FknByrsvMin9ufA8bQNHLa9meKjtWpEZFZoUqSDgHGkVIJnQDMA5wCbFxCuDVtPyPpk8AFwIGkk7rQiijP2boEuETS5qTPs4+kW4EDbV9XYDjZfkHSnsCvbB8h6ZYCjz+xwGN1rMLzovE392HgTNtPl3wT+zNgW+B2F3zV2ahoJP0WONf2+fn5VsA2RcZqcRXwbkmLAxeTLoY+ARRyMdTkadsXFHzMWanyvPgDcDVwKU0VbJ36riICPg6sC9wEYPthSYuUFGue3Gy1DfAL2/+VVPgtaO4j2hXYDfgXsC9pMvE6wJlAkVemkvRO0h/9nnlbYRnVbZ/Y/FzSwnn7c0XFmIWqzos/5SvtF4G989XpSyXEaXgQuKPoSqjFO2x/rvHE9gWSjigxXqkXQ5LWyw8vl3QkcA7wcmO/7ZuKitWkyvNiQdv/U9Kx50g/VkSv2HajQsi3+WU5GrgPuBW4StKKwDMlxLkOOBnYppF4Npso6TcFx/oK8E3SFfBkSW8BLi84BpLWIn2mJdJTPQZ8yvbkomNllZwXtg/MX9JP254h6Xlen82+SAcA50u6kpm/TH9SYIyHJR1EuoOEdJHycIHHb1XqxRDw45bnzbnYDLy3wFjpoNWeF3+S9KHGHexI0Hd9RJK+Tko19AHg+6R+ld/bPqqi+HPbnl7wMXe0Pb5l2w62zywyTsvxF7T9QonHvxb4lu3L8/PNgO/ZfldJ8Uo9LyRtO9h+2+cUEadN3IuB54DbgVeb4h1WYIwlgEOATfOmq4DDbD9ZVIyWeJsCXwf+avuH+WLoK2X2H5alyvNC0rMM9OktRLow+S8F9hvOcdn6rSICkPQB4IOk/4CLbF9S8PH3H2x/wVejSLrJ9nqz21ZQrHeSUjotbHsFSWsDX7C9T8Fxbm3tqG23reCYpZ0Xkk4YZLfL6iiWdIfttco4dq+TtB+pv/BZ4LfAeqQ+14sLjFHLeTHS9GVFVLbc8Q2p43sDBpLEfhS4wXYhwzJzp/CHgB2BM5p2jSYNlCh8EUJJfyMltp1ge928rfAvO0nnkvprTs6bdgXWt/3xIuP0utzcc2mRX55Nx/6Z7a9I+iNtRjra/ljRMXPcsaQ7ojHMPOqr0CazxoWPpC1Io18PAk4u4wKvSpI2Bm6x/XweIr4e8DPbD9RVpr7rI8q3wj8E3ki68i38trTR7CHpKmA928/m54cCfy4qDqkdfiLwMdJovIZnScNOS2H7wZYRPWWMvPkMcBipoxjSKJ/Srg6bmi0A5iWNmnu+6OYKSYsyczPWlcDhtp8uMk6TvYGvSyqjGaZxkfCjAo41FGcCvwGOpdxRX42T/EPASblPtJShbBWfF78G1s6tGV8j/R5PBt5TQqyO9F1FBBwBfNT23yuItTTwStPzV/K2Qti+FbhV0qlF9zsN4kFJ7wKcRwTuBxT+u7T9FFBZm7/t10bI5S+brYF3lBDqeOAO0l0spJGOJ5CGWBeu+XOVcOzGxc9E8rweAEmjgPnKigtMt/3rEo/fMCn3sa0EfDOPonx1Nu+ZU1WeF9PzwJytSaN5j8sjEGvTd01zkv5qu4w5Q+1ifYt0Yp2bN20DnGH7+wUdf7ztHSXdzsxNI42r3rcXEacl5pLAz0kTaEWax7Ff0ZP8qmp+mU0Zbm40PxZ4zFtsrzO7bQXG27TddttXFRjjeuD9jSH2ecj9xSUOLDkU+Dfp76p5JGChgyOUMoisA0y1/Z88TWJZ27cVGSfHquy8yCMoLyS1MLyb9Lu8pYzvi0714x3RRElnAOcx80lc+Kgl2/8n6QLSfzbAp23fXGCI/fK/HynwmIOy/TjFTxxsp6rmF+B1o5fmIg3ZLWMex4uSNrF9TY67MWnuSFm+0fR4fmBDUjNukRX6/M3zvGw/J2nBAo/fqrEIZ/NnM/CWIoM4ZW64Fxgraf4ij91GlefFJ4BdSN9Hj+aLlTKnscxWP1ZEo4EXSKOjGsxAX8SwSRrtlFFhCdI8ovua9i1R1JWb7Ufyw8dpk/KkiBitVF16kKqaXxo+2hyb9H9WxjyOvYCTcp+ASAtJ7lFCHABsN38uJC1PyrZQpOclrdeY6ClpfUqsXF1S6qBWkj5LuthbDriF1FR7HSXMI6LC8yJXPpcDu0g6BbiX4s+JIem7prkqSPqT7Y9o5lxcDbZd6JWbSs7/1hLrWtLAgdb8W2cXdPxGsswvU0HzS12UE+DaLmOC82BxBUy2vWaBx9wAOJ00eEbAMsAnmvqQiorzXtuXzWruTdGtGrnJewPgeqc8cKuT5rKV0p+XY5Z2XuSL1J3zz+OkkbZft71i0bGGqm/uiCQd4JQKpG1SzSInw9n+SP63kis3ys//1qzs9CCTmLnyLrX5ZVbnw2sBC54kKWk+YDvyHWVjEJbtw4uM0xSv+fM1+jwKTVFj+8b8Jb1a3nS3C05Amr0HuIyZ715fKwYFtmpkL9l+SRKS5rN9l6TVZv+2oavovLiLdBH5EdtTctzSRtcORd9URAyM7Ko0qaakjzEwJPMK238qJ0ypKU+alZoepMLKu6H5fDiMNIS2TH8AniZVuC/P5rVFaP5804HTbP+1hDirAWuS+qHWk4Ttk4oMYPuQ/O+nizzuIKZJWozUn3yJpKeA+0uKVcV5sS2wEymH3oWku9hSM+52KprmSiTpB6Rb+1Pzpp2BG23/b8FxKkt5kufblJ4eJA8N35umShw4uqQr7UbMwkfJtYnRc5kO8gTuzUgV0fnAVsA1trcvKd5Mdw+N7WXdVeaY7wEWBS60/crsXj8Hx6/svFDKo7g16fvovcBJpNyRhU967rhM/VIRzWr2d0MZs8Al3Qas0zK/4uY6h0l2C0nHkiaVNrJx7wbMsP3ZEmOWkhapJcYxwFG2by85TmVD+3OMtUnn9tqSlgZOsf2BomK0xLuQgbuH5n7K1mSlXaOq86JN3MWBHUh9eu+rMnazfmqaq3r2d8NipBEwkK6oClfFnBtJq+c28rZf1C4+Nf4Gnjmv3GVK6yt1u02APfJAlpcpb85XlUP7GyM2p+fO9n8Dy5cYbznbW5Z4/DpUdV7MJE8cPyb/1KZvKiLbVzYeS5oXGJufltWxCimL8815qKRIzUwHlhCnijk3+wOf5/Up8qGc1PgzJK1s+x6A3NxY+GfTzKl9FpTUGK1UVkbirQo+XlsVD+2fmPtSfku6S3mONMy5LNdKeltZdw95YEIV/XfNKjkvRqq+aZprUFpO4ETSPBGRrtx2L3KmeUu8N5H6iSAlPH20hBiTbK9f9HHrJOl9pBQnU0n/TyuSJuAVvvZR1VTRUvU5VmVD+3O8McDoMrIPNMW4E1iFNP+l8LuHRhOtpJNt71bEMTuMW9l5MdL0Y0U0CdjF9t35+VjSSKJSvsglLUv6Em1uMiu00qsq5UmOdQ0pIePVpMERzxYdoynWfMw8JLjqq9TCqWlJcttjJb2ZtDR0KWmnmr5U9wUWaAztd0kphaqgtMDk69guZESbpDuA7wHfYebpA404hWdhqfq8GGn6pmmuyTyNSgjA9j/yCK3CSfohKZ3GZAaSJZq0cFiRKkl5ku1GusLeDjhSKavz1bYLnY8g6YvAqY0ra0mLS9rT9q+KjFODKpeqh2qH9leiUeFIeiNpuHjR9iL9vhbj9XOWypivBNWfFyNKP1ZEk/KIrOZljcuaW7QN6Qqn1Cv5Kufe2L5X0kukTOKvAJsDa5QQ6nO2f9kU9ylJnwO6vSKqcql6qGhp9yrluXk/Bt5MaglYkTRP8K1FHN8p39s1kibaPq6IY3ag6vNiRJmr7gLUYC/gTlIKmS/nx3uXFGsqaQhyqSQtKOmgPAQUSatKKmW0lKR7SBP8liat1LpWSSOYRkkDa7/koe/zlhCnauMlHQ0slivWS0md/KWwfWWemnBUfj61hGwR72i+epc0WtJGRcZo8R1S3rd/5Iuw9wHXlxDnZElflnRW/tm3rNYTKj4vRpq+6iPKX2aTba9eUbyzSfMr/sLMfTdFfxGcQRqt9CnbayllPr62jH4ApeWTNyEN8riL1F90VWN0W4FxjiRd6R6dN30BeND214qMUweVvFR9S6zSl3aXdDNpAcjG1fxcwMSy5mTlO5VxeTj/unlEYOHLyFc9l63K82Kk6auKCEDSH4B9XcGyuJJ2b7fd9onttg8jTuMP87XMAGX8YbbEXBj4NGn+0nK2C+13yF9mXyBd7QJcAhxru/QlIXqJKljavd3gB0m3lTUHRtKlpGbv7wNLkprnNnDB6x+1+xsq+++qX/VjH9HiwGRJNwDPNzaWkVmh6ApnEK9IWoA8H0bSypSUr0rSj0l3RAsD1wIHk0bQFSpno/h1/ukZqmCp+lYuf2n3qZK+zMD/1T6kZumybE1aK+qrpD7eRYEy0vtUMpctH7vy82Ik6ceK6NtVBZK0KumqrZEMEgAXvAwEcChpxcXlJZ0KbEy6WynDdcARtv9VxsH1+pQ0zdwDV6NVLlUP1Sztvhfw/4CDSP93fyFNfi7LZ0gphJ5ioNmsDN8gJQidaS5bSbGqPi9GlL5pmlNaYXEv0kS424HjbE8vOeY1pGzOPyUNA/00MJftg0uI9QZSB65I66c8XnSMKsxijkhj4vE3bX+o4iIVShUuVZ/jVbK0e5UkfZeURfom4HhSf0opX2RVzWWr+rwYafqpIjqDlC36alI6jftt7zf4u4Ydc5Lt9SXdbvttzdsKjvMXtyQsbLet20hal7Sk8Q6kWfRn2/5FvaWaMxpYzO09pIXjSl+qvmyqcI2vNrFF6tj/NGki6HjSxWWhg2bK1ovnxZzop6a5NZsqg+OAGyqI+XLudP+npC8BD5H6VgqR7/IWBJZUyqLb6AgYDSxbVJwqqf0qkrK9ea0FG77miZGlLlXfTOUu7V7LGl+Q2mglPQo8SlpnaXHgLEmX2D6g6vIMQy3nxUjTT3dEM6X4b31eUswNSH+si5HmPowGjrRdyJyHPJT6K6SJfQ8xUBE9A/y2yLsHDSzh3VZR6YQkvUq6a93TA6tITi2hX60vqOSl3euQz/tPkS5UjgXOs/3fxkWf7ZVrLWAYsn6qiGYwMEpOwAKkK5DSRqdIWs/FL4/QLs6+to8qOca9DCzhvQLwVH68GPBAUdkdJG1Dav/fmDQA43TSsO2qV24tRR559XNSf55Jgz++4vKSnpaWV041rPGV4x4GHO82ueUkrVFUh7+kc0hzsC7IozhLU/V5MdL0TUVUB6XlH5YBzgLOsH1HibHexeubXwpdqjnH+S0pXcz5+flWwDa2v1BwnBG3imQRJF0P/BI4LW/aiTSvrZRMBLlj/1qXsLS70qqls+SmpVcKjvsO0sT0Z/Pz0cAatv9WcJz3k/qg3kFaauUEN+WpLDhWpefFSBMVUckkLQPsSEp+OppUIX234BgnAysDtzDQ/OIyOoubB14Mtq3gmCNiFckitJvoWeYkSVW0tHuVasjksCjpguhbwIOk1DunuMB1zKo+L0aaqIgqIultwAGkL9NCc6ZJ+jtpMEbp/5mSLiL1OTQnjd3U9hZlx+4FShnZnyI1OZp0gbI4cCSUs3RH2SqcL9eIV1kmhzwtYldSep+HgVNJE7rfZnuzAuP03HkxFFERlUjSGqQTajvgCdIIsLNt/7vgOGcCX/bAqpylyYMWDiGtNttY0uLwXv9DKUrua5sVF/3lrfZLuz9Nmr5QyDy6KufL5XjnAFcwcyaHzW1vU3Ccc0lziE4Gftf899VIq1VgrErPi5EmKqISSbqOdIVzpu2HS4xzObAOaUh68xyEUjqLc8yFbD8/+1eGOuW+h/VIk7gB3gbcQUqLs3cRfW5VzZdrivdGUiaHxvL0l5I69ou+wPtQa9+a6llGvOf10zyiytl+Z0WhDq0oTmNQxLGk+VClZHMOhXqYNBR+MoCkNUl52Q4gzVEpYvBHqfPlWuUKZ6eyjt/ku0DrII/rSBV7KFBURCWqqu28rNFJs/BTYAtgQo59q6RNK4wfhmZsoxICsH2npNVtT21JhDoc+5EmVn+ZNF/uvQysGlw4ScuR1ldqpMS5mpS2aFpBx1+GNCF8gZzdo3mi+IJFxAgzi4qoXCcw0Ha+ObntvKiDS7rG9iZ5ZFRzG2upI6NcfjbnUJzJkn5NaiKG1Gd5Z86hVsioL9s35ofPUV5S0GYnAL8njaSENJjgBOADBR1/C2APYDngJ03bnwX+t6AYoUn0EZWo6rbzKkg6i/TH+QtgI9LV8DjbVQ1I9twAABHCSURBVDSVdK1ZDBp4TVkTn5WWB9mHNNIL4K+k5dZfAha0/dwwjj1hsP0lTmhtN2qu8Im7krYrOwNFXefFSBN3ROWqtO28InuRZoAvS/o8F5O+6MLgfjzIPjPQ8V4o2y/m2O3iz3EllL2TNK/mNOBvDDRhle0JSbsyMPlzZ9Ko1EJI2tX2KcAYSfu37rf9kzZvm1ON/5f5SclbbyX9Ht9OyuFXVT9zraIiKlelbecVWc32J5s3SNqYdKUdZqGupK0l91MuQ2oO25mUJf3PwGnNfVIl+Qypj+inpEr8WoptElwo/1v6RWPjvMhD0tezfXt+vhYVDkKqWzTNhSFplyy2igSyvSR/ybRWDIWnY8qxKpnjk/ucdiZNwDzMXbpcR10kTbb91tlt61VxR1SCKpNBShoFXFr2FbekdwLvApZqaa4YDYwqM3YvkXQIsBmpIjqftDbWNaRcemVYwPZfJCknCT1U0iTSEu/DliugD5MqoTGk+T3nFnHsQWKWubRF5XGy2yQdy8wZS24rIc6IFBVROX5UVSDbMyS9KmlR20+XGGpeUlPF3MAiTdufAbYvMW6v2R5YG7jZ9qclLc3Al08ZSuunlHQSsBapQj2szKS+Lf5AGrJ9KeWO2KwqDqQ71b1JzfmQMpb8etYv7y3RNFcCSSvYfqDCeH8A1gUuYWCpi1JWyJS0otuk3w+dkXSD7Q3zXcnmpCHBf7e9eknxWtfEWhQ4wgWsiaW0dlTjfKts+kAZI+TqjBPijqgs55FnX0s62/Z2Jcc7h+pWcvydpHbLQpcy6qsHTZS0GCmD8yTSyLXrygpW5hwf24XNiRuiP7VLv9ONcSSNt72jpNtpv9x64YlcR6K4IyqBpJttr9v6uOSY8wJj89O7i0xR3xKneQ7U/KSErtPdXcszjwiSxgCjbRfeF1DXHJ8yNU3cFhUsbVHFEhqS3mT7EUkrttvfL60PcUdUDs/icSkkbQacCNxH+mNZXtLutq8qOpbtSS2b/irphqLj9CpJf3FeU8n2fa3bClTXHJ/S2F5k9q/qrniNjN6278/9hRvkXTcUncR1JIuKqBxrS3qG9Me/QH4M5bWd/xj4oPPqkZLGkr6ACs/gkJeBaJgrx1i06Di9RtL8pDllSyot9Necv2zZEkLWNcendO0q7iIr85yL765ZZT0oI9uBpB1JQ9+vIJ0bR0n6hu2zio41EkVFVALbVQ9nnsdNSxjb/oekeUqKNYmB5pHpwL3AniXF6iVfAL4CvBlo/iJ7hpQuqVC2ZwAXAhc2zfG5QlLXzvHJlflClF+Zf400bLtdNoqysmB8C9igcReUh45fCvRFRRR9RD1A0vHAq8w8B2FUSfMdwjBI2tf2URXFap3jMwE43vZDVcQvmqT9GKjMH2KgInoG+G23VrAAzfko8/O5gFubt/WyqIh6QP7C+SIDiS2vBn5V5AJekrYdbL/tqkbtdbU8qGQv0gq3kJpiji56cEnLHJ/TK5zjU7qyK/M6znVJR5LyyzXy530CuM32/xQdaySKiih0RNIJg+x23H11Js+en4c0uARgN2CG7c8WHKeWOT5VUVqgcQwzZzwoJDtFXee6pO1oWmPJdqkZKkaSqIi62KzmHjT0yxyEbiBpbtvTJd1qe+2Wfa/bFmZN0snAysAtDGQ8cBkTuEM1YrBCd/tIVYEaqfHbpcWHwlPj96IbSJOcZ0ha2fY9AJLeQiwsOFTjgDVdwVW0pA8Db2XmBLWHlxBnW+CHwBtJd609cefaqaiIuljzZLcK5iA0UuNXOpejhzQ61r8OXC5pan4+hmpWNe0ld5CGpz9SZhBJvyENud8cOJaUJ7CsOXNHAB+1/feSjj+iRdNcD2gzB+HdQN/MQegGkqYxsOz0AgxkLJ8BvBh3lJ2TdDmwDqlSeG1ATtHZIiTdZvvtTf8uDFxg+91Fxsmx/mp749m/sjfFHVFvKH0OgqT/N9j+aJ+frVGkrNetGQ5as5mH2Tu0ojgv5n9fkPRm0iqwbyop1kRJZ5DyVDZXrn0xGjUqot4wV0tT3BOkrAdFaqT22Zi0ls4Z+fkOwJ0Fx+pFj5TRt9CPbF9ZUag/5QS1R5ImIZvURFeG0cALwAebtpnqkhnXKprmesAs5iDcXkYiUknXA5vYnp6fz0MaavqOomP1kqqS3/YDSe8gLRW+BmmdrFHA82V27Oe5evOXvOZX34o7oh5g+xt51E1jQusxJc5BWJx09fZkfr5w3hYGV3RS0372C2An4EzSCLpPMZB5vjCSvgicavs/tl+WtKCkfWz/qoRY85NSZbWO0OuL+Xl1rScSCiBpFUkbQ2pLtr2/7f2BxyStXFLYHwA3S/qdpBNJTRbfKylWz7D95OxfFTplewopjdUM2ycAW5YQ5nO2/9MU8ylSDroynEwaCbgFcCWwHGnRxL4QFVF3+xkpz1arp/O+wuU/+o2Ac0nt1++0feLg7wqhUC/kVEm3SDpC0lcp57tslKTXBpdIGkVqCizDKra/TWpiPJGUI3CjkmKNONE0192Wtn1760bbt+dF10ph+1HgD2UdP4TZ2I1U8XwJ+CqwPGmBxqJdCJwh6ej8/At5WxkauQb/I2kt4FHS5Na+EIMVupikf9pedRb7pthepeoyhVCFKlYkzhmwv8BA/94lwLF5iY2iY30WOJs06OgEUt/rt20fPegbe0RURF1M0mnAZbZ/27L9s8AHbH+iwFgr2b63qOOFMKfarUgMlLIicVUkjSqjgusWURF1sZzW51zgFQbm+YwjtWN/PDehFRVrku31S1rWOoSOSZoE7NK6IrHtQlYkriOZsKQHyE2BpIvLvvpijoqoB0janLT2DMBk25eVEONm0nDZvYGftu6PFDWhKo2UO7PbNozjrzjY/uYcj0WRtCApifFOwPrAH0nrSF1TdKyRKCqi0BFJqwHbkFbI/E3rftuHVV6o0JfyekEzqHhFYkmbADvb/mLJcRYHfg580vao2b2+F0RFFIZE0la2L6i7HKF/VbEicVOsdYFdSKms7gXOKWt1WEnvIWVF2RKYCJxh++wyYo00URGFIZG0KHAIA0tdXwkcHqlPQhXyXJ7JtlcvMcZYYOf88zip3+brtgdtshtmzPuAm4HxwATbzw/+jt4SFVEYEklnk9aDaV7qem3b29ZXqtBPJP0B2Nf2AyUd/1XSXdaeOYMDkqbafksZ8fLxR9tuNzm9L8SE1jBUK9tunjx4mKRbaitN6EeLA5Ml3QC8dudQ4HpE25IGDVwu6ULgdF6/fEfRXsm57foy11xURGGoXpS0SWM0T8519+Js3hNCkb5d5sFtnwecJ2khYGvSAJ03Svo1cK7ti0sIezJwFynX3OGkARh9s1prNM2FIZG0NnASsGje9BRpMuFt9ZUq9IOcoXovYBXgduC4xnIkFcRenDRg4RNlzKNrLBPStBpsXy2vEhVRmCOSRgP0c7t2qFZewfS/pP6brYD7be9Xb6mKIekG2xtKugrYh5Rr7oYy+6VGkmiaC3MkKqBQgzVtvw1A0nHADTWXp0jH5LuubwMTSLnmDq63SNWJO6IQQleQdJPt9Wb1PHSvqIhCCF1B0gwGRskJWAB4IT92mUuFl0XS/oPt75fUWdE0F+aIpJWAdYE7bd9Vd3lC7+vRdDeL1F2AkSAqotARSefZ3iY/3pq0AuwVwPclfd/272osXgjdakHb/yNpB9tn1l2YukTTXOhIY3hpfnwtKSHjvZKWBP5ie+16SxhC98lLTrwdmNTP/V1xRxQ61XzFMndjkTzbj+eUKCGEobuQNBdvYUnNI1G7tt9rTsQdUehIU0exgPmAFW0/kpdsnljGYmEh9AtJf7C9dd3lqEtURGFYJC0GrGH7urrLEkLoTlERhRBCzSRtC/wQeCOp1SGa5kIYCkm3N2a8hxCGTtIU4KO2+ybRabMYrBA6kq/Y2u4ClqmyLCH0oH/1ayUEURGFzp0BnMrMo+ca5m+zLYTQuYk5qet5wGtLnts+p74iVScqotCp24Af2b6jdYek99dQnhB6yWhSuqIPNm0z0BcVUfQRhY5Iejcp7f7rlmeWNM72xBqKFULoAVERhRBCzSQtBxwFbJw3XQ3sZ3tafaWqzlx1FyB0B0kLSjpA0jckzS9pD0kTJB0haeG6yxdClzuBtA7Rm/PPH/O2vhB3RKEjksYDD5JS768G/J00gOFjwDK2d6uxeCF0NUm32F5ndtt6VQxWCJ0aa3tHSQIeAd5v25KuAW6tuWwhdLsnJO0KnJaf7ww8UWN5KhVNc2FInG6hz8//Np7HbXUIw/MZYEfgUdKF3vbAp2stUYXijih0aqKkhW0/Z/szjY2SVgaerbFcIXQ92/eTmrn7UvQRhWGTJMeJFMKQSTp4kN22/Z3KClOjqIhCxyS9AdgFWD1v+jtwmu2+acsOoUiSvtZm80LAnsAbbPfFiNSoiEJHJK0BXAZcBNxMyjG3LvAB4L2276qxeCF0PUmLAPuRKqHxwI9t/7veUlUjKqLQEUlnAeNtj2/Zvh2wi+3t6ilZCN1N0hLA/sAngROBn9t+qt5SVSsqotARSXfbXm2o+0IIsybpSGBb4Bjgl7afq7lItYiKKHRE0k221xvqvhDCrEl6lZRtezozT4Poq4XxYvh26NQbJe3fZruApaouTAi9wHbM5SQqotC53wKLzGLfsVUWJITQW6JpLoQQQq3itjB0TNJWkq6S9Hj+uVLSh+ouVwihu0XTXOiIpM8BXwAOABqL4I0DfiBpOdvH1Fa4EEJXi6a50BFJdwKb2H6yZfsbgGtsr1FPyUII3S6a5kKn1FoJAUR6nxDCcEVFFDr1jKS1WzfmbZF9O4Qwx6KPKHTqa8AESScAk/K2ccDuwK61lSqE0PWijyh0TNLSwBeBt+ZNd5LSkjxaX6lCCN0uKqLQEUnHABcAl9qOprgQQmGiIgodkbQRsBXwPuAV4GLgQtu31lqwEELXi4ooDFkesv1BUsX0duAmUqU0ftA3hhBCG1ERhWGTtD6wpe3/q7ssIYTuExVR6JikLYBtgGXzpoeA82xfVF+pQgjdLiqi0BFJPwPGAicB0/Lm5YBPAf+0vV9dZQshdLeoiEJHJP3D9tg22wX8w/aqNRQrhNADIrNC6NRLkjZos30D4KWqCxNC6B2RWSF0ag/g15IWYaBpbnng6bwvhBDmSDTNhSGRtAxNgxUiq0IIYbiiaS50TNKmwKK2JwHzA7vGwnghhOGKO6LQkTxqbkNSc+5FpAwLFwDvAW62/Y0aixdC6GJREYWOSJoMrAUsQJo/tKztFyTNQ6qI1qq1gCGErhVNc6FTdrpqebXxPP/7KnEehRCGIUbNhU79WdLVpL6hY4Hxkq4nNc1dVWvJQghdLZrmQsckvZN0Z3S9pJWBjwMPAGfZfnXwd4cQQntREYUQQqhVtO2HjkhaXtLpkq6W9L95kEJj33l1li2E0N2iIgqdOh64AtgXeBNwZV6XCGDFugoVQuh+MVghdGop27/Jj/eVtCtwlaSPMTCCLoQQhiwqotCpeSTNb/slANunSHqUNLl1oXqLFkLoZtE0Fzp1LLBR8wbblwI7AHfUUqIQQk+IUXMhhBBqFXdEoWOSNpd0jqTJ+ecsSZvVXa4QQneLiih0RNKHSSPn/gjsAnwSOB84PjJwhxCGI5rmQkckXQHsZ/vWlu1vB46y/Z5aChZC6HpxRxQ6tUxrJQRg+zZg6RrKE0LoEVERhU49P4f7QghhUDGPKHRqZUkT2mwX8JaqCxNC6B3RRxQ6ImnQPiDbV1ZVlhBCb4mKKAyJpPmBVfLTKY1MCyGEMKeijyh0RNLcko4ApgEnAicBD0o6ojkTdwghDFVURKFTRwJLACvZXt/2esDKwGLAj2otWQihq0XTXOiIpH8CY91ywkgaBdxle9V6ShZC6HZxRxQ65dZKKG+cQSwDEUIYhqiIQqfulPSp1o15XaK7aihPCKFHRNNc6IikZYFzgBeBSXnzOGAB4OO2H6qrbCGE7hYVURgSSe8F3pqf3mn7L3WWJ4TQ/aIiCiGEUKvoIwohhFCrqIhCCCHUKiqiEEIItYqKKIQQQq3+Pwq5Ntbv5RIsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "feat_labels = df_wine.columns[1:]\n",
    "\n",
    "forest = RandomForestClassifier(n_estimators=500,\n",
    "                                random_state=1)\n",
    "\n",
    "forest.fit(X_train, y_train)\n",
    "importances = forest.feature_importances_\n",
    "\n",
    "indices = np.argsort(importances)[::-1]\n",
    "\n",
    "for f in range(X_train.shape[1]):\n",
    "    print(\"%2d) %-*s %f\" % (f + 1, 30, \n",
    "                            feat_labels[indices[f]], \n",
    "                            importances[indices[f]]))\n",
    "\n",
    "plt.title('Feature Importance')\n",
    "plt.bar(range(X_train.shape[1]), \n",
    "        importances[indices],\n",
    "        align='center')\n",
    "\n",
    "plt.xticks(range(X_train.shape[1]), \n",
    "           feat_labels[indices], rotation=90)\n",
    "plt.xlim([-1, X_train.shape[1]])\n",
    "plt.tight_layout()\n",
    "# plt.savefig('images/04_09.png', dpi=300)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IT9KXr1cMq2q",
    "outputId": "0f671b24-992b-48b0-e373-5eb51fa7c63d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "이 임계 조건을 만족하는 샘플의 수: 5\n"
     ]
    }
   ],
   "source": [
    "from sklearn.feature_selection import SelectFromModel\n",
    "\n",
    "sfm = SelectFromModel(forest, threshold=0.1, prefit=True)\n",
    "X_selected = sfm.transform(X_train)\n",
    "print('이 임계 조건을 만족하는 샘플의 수:', \n",
    "      X_selected.shape[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G5TgJmv0Mq2q"
   },
   "source": [
    "앞서 지정한 특성 선택 임계 조건에 맞는 특성 5개를 출력해 보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YxFBwmc2Mq2r",
    "outputId": "0ccbd598-294e-4b96-9635-4c7fffbaab84"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 1) Proline                        0.185453\n",
      " 2) Flavanoids                     0.174751\n",
      " 3) Color intensity                0.143920\n",
      " 4) OD280/OD315 of diluted wines   0.136162\n",
      " 5) Alcohol                        0.118529\n"
     ]
    }
   ],
   "source": [
    "for f in range(X_selected.shape[1]):\n",
    "    print(\"%2d) %-*s %f\" % (f + 1, 30, \n",
    "                            feat_labels[indices[f]], \n",
    "                            importances[indices[f]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WPRM3RfIMq2r"
   },
   "source": [
    "`RFE`는 재귀적 특성 제거 방법을 사용합니다. 처음에 모든 특성을 사용해 모델을 만들고 특성 중요도가 가장 낮은 특성을 제거합니다. 그다음 제외된 특성을 빼고 나머지 특성으로 새로운 모델을 만듭니다. 이런 식으로 미리 정의한 특성 개수가 남을 때까지 계속합니다.\n",
    "\n",
    "`RFE` 클래스의 `n_features_to_select` 매개변수에 선택할 특성의 개수를 지정합니다. 사이킷런 0.24 버전부터 [0, 1] 범위의 실수를 지정하여 선택할 특성의 비율을 지정할 수도 있습니다. 기본값은 입력 특성 개수의 절반입니다.\n",
    "\n",
    "`step` 매개변수에서 각 반복에서 제거할 특성의 개수를 지정합니다. (0, 1) 사이의 값을 지정하면 삭제할 특성의 비율이 됩니다. 기본값은 1입니다.\n",
    "\n",
    "기본적으로 기반 모델의 `coef_`나 `feature_importances_` 속성을 기준으로 특성을 제거합니다. 사이킷런 0.24 버전에서 사용할 속성을 지정할 수 있는 `importance_getter` 매개변수가 추가되었습니다.\n",
    "\n",
    "`SelectFromModel`과 비교하기 위해 5개의 특성을 골라 보겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "M77_9gz0Mq2r",
    "outputId": "f31f6be2-2dd6-4e54-c001-65b7e9c9e8b8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RFE(estimator=RandomForestClassifier(n_estimators=500, random_state=1),\n",
       "    n_features_to_select=5)"
      ]
     },
     "execution_count": 86,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_selection import RFE\n",
    "\n",
    "rfe = RFE(forest, n_features_to_select=5)\n",
    "rfe.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N2ILFBJ9Mq2r"
   },
   "source": [
    "`ranking_` 속성에는 선택한 특성의 우선순위가 들어있습니다. 1은 기반 모델이 선택한 5개 특성입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KU6wOcZCMq2r",
    "outputId": "ce1fdcf5-1177-4b1a-fb12-765137c52921"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 5, 9, 6, 4, 3, 1, 8, 7, 1, 2, 1, 1])"
      ]
     },
     "execution_count": 87,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rfe.ranking_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iiGyqrZkMq2r"
   },
   "source": [
    "선택된 특성은 `support_` 속성에 `True`로 표시됩니다. 이 값을 `f_mask`로 저장하여 특성의 이름을 표시하는데 사용하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "id": "-VoV7HvoMq2r"
   },
   "outputs": [],
   "source": [
    "f_mask = rfe.support_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_P4jp4Z6Mq2s"
   },
   "source": [
    "훈련된 기반 모델(랜덤 포레스트)은 `estimator_` 속성에 저장되어 있습니다. 이 모델은 `n_features_to_select`개의 특성을 사용하여 훈련되어 있습니다. 이 모델의 특성 중요도를 저장하여 앞에서와 같이 중요도 순서대로 출력해 보겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "id": "ChTkVovNMq2s"
   },
   "outputs": [],
   "source": [
    "importances = rfe.estimator_.feature_importances_\n",
    "indices = np.argsort(importances)[::-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bIqrHXRmMq2s"
   },
   "source": [
    "기반 모델을 동일한 랜덤 포레스로 사용했기 때문에 `SelectFromModel` 클래스로 선택한 특성과 순서가 약간 바뀌었을 뿐 동일합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IijMK5PaMq2s",
    "outputId": "7b3257fc-2089-42ba-ccc3-c3b763f72df4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 5) Proline                        0.261512\n",
      " 5) Color intensity                0.216477\n",
      " 5) Flavanoids                     0.212259\n",
      " 5) OD280/OD315 of diluted wines   0.188924\n",
      " 5) Alcohol                        0.120828\n"
     ]
    }
   ],
   "source": [
    "for i in indices:\n",
    "    print(\"%2d) %-*s %f\" % (f + 1, 30, \n",
    "                            feat_labels[f_mask][i], \n",
    "                            importances[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YERlQ0q5Mq2s"
   },
   "source": [
    "<br>\n",
    "<br>"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "name": "ch04.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}