eur-jpy.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "colab": {
      "name": "StatsFinal-2.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "AOiKCJJRZbEX",
        "COubiYX8jUBC",
        "xRk2SjnapGHg",
        "R4Ph4x12snF9",
        "QRAxK5lM4Lli"
      ]
    },
    "language_info": {
      "name": "python"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "31Qf7W12bYnV"
      },
      "source": [
        "We would like to know the sample variance of the daily changes in value against the USD for the Euro\n",
        "(EUR) and the Japanese Yen (JPY). We also want to find the p-value for the hypothesis that EUR and JPY have the same volatility.\n",
        "\n",
        "To start off this problem, we read the files and prepare datasets as usual:"
      ],
      "id": "31Qf7W12bYnV"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DmDNbbjZ4TfW"
      },
      "source": [
        "# Read the csv files\n",
        "if os.path.exists(\"EURtoUSD.csv\"): \n",
        "    eur = pd.read_csv(\"EURtoUSD.csv\")\n",
        "if os.path.exists(\"USDtoJPY.csv\"): \n",
        "    jpy = pd.read_csv(\"USDtoJPY.csv\")"
      ],
      "id": "DmDNbbjZ4TfW",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZYUj5BKk5nto"
      },
      "source": [
        "# Prepare data\n",
        "eur['Change %'] = eur['Change %'].str.rstrip('%').astype('float') \n",
        "jpy['Change %'] = jpy['Change %'].str.rstrip('%').astype('float') "
      ],
      "id": "ZYUj5BKk5nto",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AxCjRm4nAwKv"
      },
      "source": [
        "x = eur['Change %'].to_numpy()\n",
        "y = jpy['Change %'].drop([8229, 8230, 8231, 8232]).to_numpy()"
      ],
      "id": "AxCjRm4nAwKv",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dlKLjepEbxo8"
      },
      "source": [
        "Now, it's time to calculate the variances. There are many situations where we might want to choose the process with smaller variability for a variable of interest. This is why finding and comparing the variances is important. One thing to consider here is that we stripped data out of its percentage signs for the compatibility with numpy, therefore the variance we find will be divided by $10^4$."
      ],
      "id": "dlKLjepEbxo8"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t--5PMoO55bz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f407a0a9-3c54-44fb-988b-7068bff006cb"
      },
      "source": [
        "var_x = np.var(x, ddof=1)\n",
        "var_y = np.var(y, ddof=1)\n",
        "\n",
        "print(var_x / (10 * 10 * 10 * 10))\n",
        "print(var_y / (10 * 10 * 10 * 10))"
      ],
      "id": "t--5PMoO55bz",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3.587397931854963e-05\n",
            "4.37880288952247e-05\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2LzClQ1xc3SY"
      },
      "source": [
        "In order to use the F-test, the samples must come from a normal distribution. The Central Limit Theorem applies to sample means, as it did in the previous question, not to the data. Hence, the fact that the sample size is large solely does not mean we can assume the data come from a normal distribution. For this example, we will assume that they are normally distributed."
      ],
      "id": "2LzClQ1xc3SY"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ksyprJ2Q_atp"
      },
      "source": [
        "In order to compare the variances, the hypotheses are as following:\n",
        "\n",
        "$H_0 : \\sigma_x / \\sigma_y = 1$\n",
        "\n",
        "$H_1: \\sigma_x / \\sigma_y \\neq 1 $ "
      ],
      "id": "ksyprJ2Q_atp"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iryvvExofM3W"
      },
      "source": [
        "The inequality indicates a two-tailed test. The test statistic F will be found by the formula:\n",
        "\n",
        "$F = s_1^2 / s_2^2$ \n",
        "\n",
        "where $s_1^2$ and $s_2^2$ are sample variances of x and y respectively.\n",
        "\n",
        "The hypothesis that the two variances are equal will be rejected if:\n",
        "\n",
        "$F > F_{α/2,N_1−1,N_2−1}$\n",
        "\n",
        "where $F_{α, N_1-1, N_2-1}$ is the critical value of the F distribution with $N_1-1$ and $N_2-1$ degrees of freedom and a significance level of $α$."
      ],
      "id": "iryvvExofM3W"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WvUbQyMQh6uA"
      },
      "source": [
        "**Regarding the following calculation**\n",
        "\n",
        "This function works only when the first sample variance is larger than the second sample variance. Because of this, we defined the two samples in a way that they work with the function."
      ],
      "id": "WvUbQyMQh6uA"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XrXAbetchniu"
      },
      "source": [
        "The F test statistic is calculated as $F = s_1^2 / s_2^2$ . By default, numpy.var calculates the population variance, that's why our code uses var(x) and var(y) to calculate F."
      ],
      "id": "XrXAbetchniu"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nQOYejApgmXs",
        "outputId": "79e9e287-bed4-40a5-9d14-50d139d2ce52"
      },
      "source": [
        "#calculate the test statistic\n",
        "F = np.var(y) / np.var(x)\n",
        "print(F)"
      ],
      "id": "nQOYejApgmXs",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.220665945827109\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LTISXLmfh6EG"
      },
      "source": [
        "Now we will find the p-value. It will correspond to 1 – cdf of the F distribution with numerator degrees of freedom $N_2-1$ and denominator degrees of freedom $N_1-1$."
      ],
      "id": "LTISXLmfh6EG"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IjSvp7zRjCTw",
        "outputId": "f7c1ff85-d7e6-453d-92cf-c658fe436c71"
      },
      "source": [
        "#calculate the p value\n",
        "d_f1 = y.size - 1 #numerator \n",
        "d_f2 = x.size - 1 #denominator \n",
        "p = 1 - scipy.stats.f.cdf(F, d_f1, d_f2) \n",
        "print(p)"
      ],
      "id": "IjSvp7zRjCTw",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.1102230246251565e-16\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3nXFk6MujBEY"
      },
      "source": [
        "The F test statistic is 1.22067 and the corresponding p-value is 1.110223e-16. Since this p-value is less than .05, we should reject the null hypothesis. This means we have enough evidence to say that the two population variances are not equal."
      ],
      "id": "3nXFk6MujBEY"
    }
  ]
}