FMML-22-M4-Lab3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/lakshmilikithabonthu/Fmml/blob/main/FMML-22-M4-Lab3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# FOUNDATIONS OF MODERN MACHINE LEARNING, IIIT Hyderabad\n",
        "# Module 4: Perceptron and Gradient Descent\n",
        "## Lab 3: Gradient Descent\n",
        "### Module Coordinator: Jashn Arora\n",
        "\n",
        "Gradient descent is a very important algorithm to understand, as it underpins many of the more advanced algorithms used in Machine Learning and Deep Learning.\n",
        "\n",
        "A brief overview of the algorithm is\n",
        "\n",
        "\n",
        "*   start with a random initialization of the solution.\n",
        "*   incrementally change the solution by moving in the direction of negative gradient of the objective function.\n",
        "*   repeat the previous step until some convergence criteria is met.\n",
        "\n",
        "The key equation for change in weight is:\n",
        "$$w^{k+1} \\leftarrow w^k - \\eta \\Delta J$$\n",
        "\n",
        "In this lab, we will discuss stochastic gradient descent, mini-batch gradient descent and batch gradient descent.\n"
      ],
      "metadata": {
        "id": "XYxxkQg6xCjD"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fr-MnaGs7JmZ"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7ob_zZms7VOu"
      },
      "source": [
        "np.random.seed(42)"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v4Kix4bcChiy"
      },
      "source": [
        "# Creating the Data\n",
        "\n",
        "Let's generate some data with:\n",
        "\\begin{equation} y_0= 4 \\end{equation} \n",
        "\\begin{equation} y_1= 3 \\end{equation} \n",
        "\n",
        "and also add some noise to the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MtAS7eFZ9hX6"
      },
      "source": [
        "X = 2 * np.random.rand(100, 1)\n",
        "y = 4 + 3 * X + np.random.randn(100, 1)"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zD95NaF-CxM-"
      },
      "source": [
        "Let's also plot the data we just created"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "id": "3IiEP4BQ7Wja",
        "outputId": "71ef8e1e-f865-46d5-8eb5-f641d1a45884"
      },
      "source": [
        "plt.plot(X, y,'b.')\n",
        "plt.xlabel('x')\n",
        "plt.ylabel('y', rotation=0)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'y')"
            ]
          },
          "metadata": {},
          "execution_count": 4
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEGCAYAAACevtWaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZbElEQVR4nO3dfbAkVXnH8e/D5U2QUti9AkHWhSpLo1RUMrFcomYFE3V9IZpoQWKxKGZjogSSyotblGJBkjWWVWLKVIUrbAKlAQxKohgJRNhQhsuauwiCIAoIuETdK4hKjCyuT/7oHhmGebszfbrPOf37VG3duT0902f69j7nzHNe2twdERHJy15NF0BERKqn4C4ikiEFdxGRDCm4i4hkSMFdRCRDezddgK7Vq1f72rVrmy6GiEhSduzY8T13n+/fHk1wX7t2LUtLS00XQ0QkKWZ236DtSsuIiGRIwV1EJEMK7iIiGVJwFxHJkIK7iEiGFNxFRDI0U3A3s61mtsvMbuvZ9mYz+6qZ/czMOrMXUURkNouLsGVL8bMtZh3n/o/AR4GLe7bdBrwJOH/G9xYRmdniIpxwAuzeDfvuC1/4Aqxb13Spwpup5e7u1wMP9W27w93vnKlUIiIV2batCOx79hQ/t21rukT1aDTnbmabzGzJzJaWl5ebLIqIZGr9+qLFPjdX/Fy/vukS1aPR5QfcfQFYAOh0OrollIhUbt26IhWzbVsR2NuQkoGI1pYREQll3br2BPUuDYUUEcnQrEMhLwEWgeeY2U4zO83M3mhmO4F1wOfM7N+rKKiIiExuprSMu5885KkrZnlfERGZjdIyIiINCTm5Sh2qIiINCD25Si13EZEGhJ5cpeAuItKA0JOrlJYREWlA6MlVCu4iIg0JOblKaRkRkQwpuIuIZEjBXUQkQwruIiIZUnAXkdbL8TZ8Gi0jIq2W62341HIXkVYbNVM05Ra9Wu4iMrHFxfzuaNSdKdptuXdniqbeoldwF5GJpB7shhk2U3RQi36Wz1t3xajgLiITqTrYxWTQTNFhLfppNFExKriLyESqDHYpqHLtlyYqxpmCu5ltBV4H7HL3Y8pthwCXAWuBe4G3uPv3ZyumiDQt9EJXMapq7ZcmKkZz9+lfbPZy4BHg4p7g/kHgIXf/gJm9BzjY3f9i3Ht1Oh1fWlqauiwiIjELlXM3sx3u3unfPus9VK83s7V9m08E1pePLwK2AWODu4hIzkKuADlIiHHuh7r7t8vH3wEOHbajmW0ysyUzW1peXg5QFBGRdgo6icmLnM/QvI+7L7h7x9078/PzIYsiItIqIYL7d83scIDy564AxxARkRFCBPfPABvLxxuBfw1wDBFpuZSXBqjDrEMhL6HoPF1tZjuBs4EPAJ80s9OA+4C3zFpIEZFesc2WjXFZhllHy5w85KkTZnlfEZFRYpotG1tF06VVIUUkOd1JQXNzzc+WHbWqZJMU3EUkOd3Zsuee23xLub+iWbUqjr4ArS0jIkmqe1IQDM6t9y7LsGoVnHlmHCkaBXcRmUmMnYkhjMqtdyuaLVvi6QtQcBdJRIxBNNbOxBAm6cSNaeVMBXeRBMQaRGMatRLaJIE7ppUzFdxFEhBrEI2ppRrapIG7ib6AQRTcRRIQaxCNqaVah0kCdyzpMwV3kQTEHERjaanGIKb0mYK7SCIUROMXU/pMk5hERCoS08xZtdxFRCoSU/pMwV1EkhBLR+U4saTPFNxFJHoxdVSmQjl3EYlerCsvxkzBXUSiF1NHZSqCpWXM7Azg9wADPubu54U6lojkLaaOylQECe5mdgxFYH8xsBu4ysyudPe7QhxPRPLT34EaS0dlKkK13H8R2O7uPwYws/8E3gR8MNDxRCQj6kCdXaic+23Ay8xslZkdAGwAjuzfycw2mdmSmS0tLy8HKoqIpEYdqLMLEtzd/Q7gb4CrgauAm4E9A/ZbcPeOu3fm5+dDFEVEEtRUB+riYhy3yKtCsA5Vd78QuBDAzP4a2BnqWCKSl/5b13Vb7iFTM6FTQXVPwgo5WuYZ7r7LzNZQ5NtfEupYIpKfbgCsK/cectGvJvoQQo5z/5SZ3Q58FniXuz8c8FgikqE6c+8hU0FN9CGETMu8LNR7i0gcQqca6rxJScix9E3cbMXcPfxRJtDpdHxpaanpYojIhOpKNVRVgTS98Fio45vZDnfv9G/XwmEiMpW6bkxRxeSlGMbN1z0JS2vLiMhUUlrvpY3j5tVyFwmk6TRAaCmt9xLrDcZDUnAXCSCGNEAdUlnvJaWKqCoK7iIBxHSj5CqM+haSyjeUVCqiqii4iwSQUxpg1LeQGL6hpFK51E3BXSSAnNIAo76FNP0NJYbKJVYK7iKB5JIGGPUtpOlvKE1XLjFTcBeRkUZ9C5n0G0qo1EnTlUvMNENVRIKaJHUyS/Bve85dM1RFpBHjUiez5s1zSX9VTTNURRKWws0lxs1kbePs0Tqo5S6SqFRGiozLyytvHoaCu0iiUhopMip1ktOw0ZgouIskKvUWb39HqIJ6tULeZu+PgXcADtwKvM3dfxLqeCJtk3KLN5WUUsqCdKia2RHAHwEddz8GmANOCnEskTZbtw42b04vMKoTNbyQo2X2Bp5iZnsDBwD/E/BYIq2QwuiYSaS0FnyqgqRl3P0BM/sQcD/wf8DV7n51/35mtgnYBLBmzZoQRRHJRm6pjI0bi5+nnJL254hVqLTMwcCJwFHALwAHmtlb+/dz9wV377h7Z35+PkRRRLIROpVR17eCbiX1sY/BRReFPVYsmvjGFapD9ZXAN919GcDMPg0cB3w80PFEolTl1PiQo2Pq/FaQ0hDOKjT1jStUcL8feImZHUCRljkB0MIx0ipV/6cOOTqmzoCb+hDOlWqqMguVc99uZpcDNwE/Bb4MLIQ4lkisQvynDjUevM6Am/IQzmk0VZkFG+fu7mcDZ4d6f5HYpdRCHRdw277y4iyaqsy05K9IQDkExarTS7mN+mmalvwVaUAO0+qrTi+1rUO1KVryV0RGqnrCkSYw1UMtdxEZqeqccds6VJuinLuISMKG5dyVlpHK5LLuiUgOlJaRSmgEhEhc1HKXSmgJV5G4KLhLJTQColq5p7hy/3wxGJmWMbNzgIfc/bzy978Cdrn7R2oomyREIyCqk3uKK/fPF4txLfetwCkAZrYXxd2UtLJjjVJq4aR6V6DY5J7iyv3zxWJky93d7zWzB83sRcChwJfd/cF6iiaxt3BymFoP8X2OlNakmUbuny8Wk4yWuQA4FTiMoiUvNYl5mnbsFc+kYvwcuae4cv98sZgkuF8BnAPsA/xO2OJIr5hbODFXPCsR6+fIYU2aUXL/fDEYG9zdfbeZXQc87O57aiiTlGJu4cRc8axELp9DpN/Y5QfKjtSbgDe7+zdCFUTLD6Qntlz1tHL5HNJOUy35a2bPA64ErlhJYDez5wCX9Ww6Gnhfd0hl6hQMCrl8tZ71c6R0PaRUVpnNuNEyt1ME5hVx9zuBFwKY2RzwAEXuPnkxdsBJc1K6HlIqq8yujhmqJwB3u/t9NRwrOI3RlV6xXQ+j5kXEVlYJq46Fw04CLqnhOLVQB1w66khBxHQ9jGuZx1RWCS9ocDezfYE3AJuHPL8J2ASwZs2akEWpTMwjWNquN5hDPSmImK6HccM6YyqrhBe65f4a4CZ3/+6gJ919AViAYrRM4LJUJpeOxJz0t1o3bqxv/Hos18MkLfNYyirhhQ7uJ5NRSkbi1d9qhfpSELGMQFHLXHoFu82emR0I3A8c7e4/GLe/xrnLLAblmyF8oKtjBEoslYfEaapx7rNw9/8FVoV6f4lPk0FoWKs1dDlCL1+g4YsyLd1mTyoRQxBqIp8cegRKrGvfSPwU3KUSbQ1CofPcGr4o01Jwl0q0OQit5BvDSlNX6iSVaSm4SyUUhMabNnWl4YsyDQV3qYyC0GhtTV1JM+pYW0ZEeDx1NTfXvtSV1E8td5GaKHUldVJwl0a0dWKOUldSFwV3qV0MY+JFcqece4uNWvs7JK0rLhKeWu4t1WTrediY+LamakRCUHBvqSaH5Q3qWEwtVaOKSGKn4N5STc8o7e9YTGkMeGoVkbSTcu4t1W09n3tuHMGpqjHgdfQjqM9AUqCWe4vFNCyvijHgIVrUg9IvTX/rEZmEgrtEY9bKpurUzrDKQpORJAUK7hFTp93KVN2iHlVZxPStR2SQYMHdzJ4OXAAcAzjwdneveUR1utRpt3JVt6hjSL+ogpdphWy5fwS4yt1/28z2BQ4IeKzspDR6JCZVtqibTr+ogpdZBAnuZvY04OXAqQDuvhvYHeJYuYqh1ViH2FumTaZfVMHLLEK13I8CloF/MLMXADuAM8qbZv+cmW0CNgGsWbMmUFHSFEOrMfSx1TIdrS0VvIQRKrjvDRwLnO7u283sI8B7gPf27uTuC8ACQKfT8UBlSVZTrcZxQbeqwK+W6WhNV/CStlDBfSew0923l79fThHcoxN7WqAJo4Jula1ttUzH06gcmVaQ4O7u3zGzb5nZc9z9TuAE4PYQx5pFDmmBEJXTqKBbZWs7hZapKn9JVcjRMqcDnyhHytwDvC3gsaaSelogVOXUH3ShmNK/fn31re2YW6Y5VP7SXsGCu7vfDHRCvX8VUk8LhKycukF3UICLvbVdlUnOr1r2EqtWz1BNIS0wSh2V06AAt3lzeudqGuPOr1r2ErNWB3eIOy3Qr7+VWEfllPq3m1mMO7+pp/Ukb60P7qkYtYhVyICS+rebWY06v3VUfEr7yLQU3BPR9J2TmhpvH3NgC13xKe0js1Bwr1DIYNS29EgqgS1kxae0j8xCwX1G3YC+ahWceWa4YNS29IgCW/sqdKmWgvsMeluXe+1VBKKf/SxcMEqp83dWCmztq9ClWgruM+htXboXAd6svcGoSgpshTZV6FKtVgT3ULnw/tbleefBgw+2OxhVSYFNZHrZB/eQHXNqXaYn9hE4IlXJPriH7pibpXUZW6CJrTxVS2UEjkgVsg/usXbMxRZoYitPCBqBI22SfXCPNXUyKNB0tzdRzjYEvlgrepEQsg/uEGfHXH+gWbXqyS1nqC/YtyHwxVrRi4TQiuAeo/5A099yvvhiuOii+tIkKwl8KefmY6zoRUJQcK/JoIDYH2h6W85Qf5pkksDXhty8SA6CBXczuxf4EbAH+Km713bjjmlaliFbo5MExEF3P+ptuceSJmlDbl4kB6Fb7q9w9+8FPsYTTNOyDN0anTQg9recY8wPtyE3L5KD7NIy07QsQ7dGpw2IMeaH1SkpkoaQwd2Bq83MgfPdfaF/BzPbBGwCWLNmTSUHnSaQhm6N5hYQY6x0ROSJzN3DvLHZEe7+gJk9A7gGON3drx+2f6fT8aWlpUqOHVvOPVfTnjOda5HqmNmOQX2awYJ738HfDzzi7h8atk+VwT1nsQTGafspNNpGpFrDgvtegQ52oJkd1H0M/AZwW4hjtUk3ML73vcXPxcXmyjJshm2o14nIygQJ7sChwBfN7BbgS8Dn3P2qQMdqjZgCY7efYm7uyf0Ui4uwZcvgymfU60SkOkE6VN39HuAFId47JytNscQ0DHFYJ/G4tEtuncsiscpuKOQkxgXVxcVi+j/AKaeE6SycJvccW2AcNGpmkmGlGm0jEl7rgvuooNoN6hdeCI89VmzbunXyce8rCdjTjq2PPTDG9O1CpM1C5dyjNSxv3Q3M55//eGCH4nGIzsJcc8/dbxfnnquRMCJNal3LfVjLshuY+0eG7rPP5IF3Ja3W2FIsVYr924VIGyQf3FfaKTksqPYG5rk52LABDjtsZTn3lQZsBUERCaWWSUyTmGYSU9UTYmKZICQiMqlhk5iSbrlXveCXWtIikoukO1SnnUgjIpK7pFvu006kERHJXdLBHaafSCMikrOk0zLD5DqGvClKcYmkJ/mW+yA5jyGvm1JcImnKMriDRr5URSkukTRlmZaR6ijFJZKmbFvuUg2luETSpOAuYynFJZKeoGkZM5szsy+b2ZUhjyMiIk8UOud+BnBH4GOIiEifYMHdzJ4JvBa4INQxYqKx4CISk5A59/OAPwcOCniMKGgsuIjEJkjL3cxeB+xy9x1j9ttkZktmtrS8vByiKLVYyR2YRETqECot86vAG8zsXuBS4Hgz+3j/Tu6+4O4dd+/Mz88HKkp4dY4FV/pHRCYRJC3j7puBzQBmth74U3d/a4hjxaCuseBK/4jIpDTOvSJ1jAXXUgAiMqngwd3dtwHbQh+nDVZyA24RaTe13BOipQBEZFIK7onRUgAiMgmtCikikiEFdxGRDCm4i4hkSMFdRCRDCu4T0sxQEUmJRstMQDNDRSQ1arlPQAuDiUhqWhfcp0mv6CbRIpKaVqVlpk2vaGaoiKSmVcF9loW3NDNURFLSqrRMk+kVjbYRkTq1quXeVHpFo21EpG6tCu7QTHpF67CLSN1alZZpikbbiEjdWtdyb4JG24hI3YIEdzPbH7ge2K88xuXufnaIY6VCo21EpE6hWu6PAse7+yNmtg/wRTP7vLvfGOh4IiLSI0hwd3cHHil/3af85yGOJSIiTxasQ9XM5szsZmAXcI27bx+wzyYzWzKzpeXl5VBFERFpnWDB3d33uPsLgWcCLzazYwbss+DuHXfvzM/PhyqKiEjrBB8K6e4PA9cBrw59LBERKQQJ7mY2b2ZPLx8/Bfh14GshjiUiIk9mRd9nxW9q9kvARcAcRQXySXc/Z8xrloH7Vnio1cD3pipkeCrbdGItW6zlApVtWrmU7Vnu/qS8dpDgXhczW3L3TtPlGERlm06sZYu1XKCyTSv3smn5ARGRDCm4i4hkKPXgvtB0AUZQ2aYTa9liLReobNPKumxJ59xFRGSw1FvuIiIygIK7iEiGog3uZvZqM7vTzO4ys/cMeH4/M7usfH67ma3teW5zuf1OM3tVA2X7EzO73cy+YmZfMLNn9Ty3x8xuLv99puZynWpmyz3Hf0fPcxvN7Bvlv41VlmvCsn24p1xfN7OHe54Lds7K999qZrvM7LYhz5uZ/W1Z9q+Y2bE9zwU7bxOU63fL8txqZjeY2Qt6nru33H6zmS1VWa4Jy7bezH7Q83d7X89zI6+FGsr2Zz3luq28vg4pnwt23szsSDO7rowNXzWzMwbsU9215u7R/aOY/HQ3cDSwL3AL8Ly+ff4Q+Pvy8UnAZeXj55X77wccVb7PXM1lewVwQPn4D7plK39/pMFzdirw0QGvPQS4p/x5cPn44DrL1rf/6cDW0Oes5/1fDhwL3Dbk+Q3A5wEDXgJsr+m8jSvXcd3jAa/plqv8/V5gdYPnbD1w5azXQoiy9e37euDaOs4bcDhwbPn4IODrA/6PVnatxdpyfzFwl7vf4+67gUuBE/v2OZFiFizA5cAJZmbl9kvd/VF3/yZwV/l+tZXN3a9z9x+Xv95IsXhaaJOcs2FeRbFy50Pu/n3gGqpdC2ilZTsZuKTC44/k7tcDD43Y5UTgYi/cCDzdzA4n8HkbVy53v6E8LtR3nXWPPe6cDTPLdRqibLVda+7+bXe/qXz8I+AO4Ii+3Sq71mIN7kcA3+r5fSdPPgk/38fdfwr8AFg14WtDl63XaRQ1cdf+VixzfKOZ/WYD5fqt8uve5WZ25ApfG7pslCmso4BrezaHOmeTGlb+0OdtJfqvMweuNrMdZrapoTKtM7NbzOzzZvb8cls058zMDqAIkJ/q2VzLebMijfwioH8p9MquNd1DNSAzeyvQAX6tZ/Oz3P0BMzsauNbMbnX3u2sq0meBS9z9UTP7fYpvPsfXdOxJnURxW8Y9PduaPGfRM7NXUAT3l/Zsfml5zp4BXGNmXytbtHW5ieLv9oiZbQD+BXh2jcefxOuB/3L33lZ+8PNmZk+lqFDOdPcfVvnevWJtuT8AHNnz+zPLbQP3MbO9gacBD0742tBlw8xeCZwFvMHdH+1ud/cHyp/3ANsoau9ayuXuD/aU5QLglyd9beiy9TiJvq/JAc/ZpIaVP/R5G8uKRfouAE509we723vO2S7gCqpNTY7l7j9090fKx/8G7GNmq4ngnPUYda0FOW9W3Hb0U8An3P3TA3ap7loL0XFQQcfD3hQdBkfxeKfL8/v2eRdP7FD9ZPn4+TyxQ/Uequ1QnaRsL6LoNHp23/aDgf3Kx6uBb1BRZ9KE5Tq85/EbgRv98c6ab5blO7h8fEid56zc77kUHVpWxznrO/ZahncOvpYndnJ9qY7zNkG51lD0KR3Xt/1A4KCexzcAr675nB3W/TtSBMj7y/M30bUQsmzl80+jyMsfWNd5Kz//xcB5I/ap7Fqr9IRW/MfZQNGbfDdwVrntHIqWMMD+wD+XF/eXgKN7XntW+bo7gdc0ULb/AL4L3Fz++0y5/Tjg1vKCvhU4reZybQG+Wh7/OuC5Pa99e3ku7wLeVvc5K39/P/CBvtcFPWflMS4Bvg08RpHLPA14J/DO8nkD/q4s+61Ap47zNkG5LgC+33OdLZXbjy7P1y3l3/usBs7Zu3uutRvpqYAGXQt1lq3c51SKgRe9rwt63ijSZg58pedvtiHUtablB0REMhRrzl1ERGag4C4ikiEFdxGRDCm4i4hkSMFdRCRDCu4iIhlScBcRyZCCu8gQZvYr5SJr+5vZgeUa3Mc0XS6RSWgSk8gIZvaXFLOhnwLsdPctDRdJZCIK7iIjmNm+wH8DP6GYQr9nzEtEoqC0jMhoq4CnUtw5Z/+GyyIyMbXcRUYo79l6KcUqhoe7+7sbLpLIRHSzDpEhzOwU4DF3/yczmwNuMLPj3f3aca8VaZpa7iIiGVLOXUQkQwruIiIZUnAXEcmQgruISIYU3EVEMqTgLiKSIQV3EZEM/T9VWltMiKM2xwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ScwxpouoDDyZ"
      },
      "source": [
        "## Cost Function\n",
        "\n",
        "The equation for calculating cost function is as shown below. The cost function is only for linear regression. For other algorithms, the cost function will be different and the gradients would have to be derived from the cost functions\n",
        "\n",
        "\\begin{equation}\n",
        "J(y_{pred}) = \\frac{1}{2} m \\sum_{i=1}^{m} (h(y_{pred})^{(i)} - y^{(i)})^2 \n",
        "\\end{equation}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PUeTUAXH7ZaV"
      },
      "source": [
        "def cal_cost(y_pred, X, y):\n",
        "    '''\n",
        "    Calculates the cost for given X and Y.\n",
        "    y_pred = Vector of y_preds \n",
        "    X = Row of X's np.zeros((2, j))\n",
        "    y = Actual y's np.zeros((2, 1))\n",
        "    \n",
        "    where:\n",
        "        j is the no of features\n",
        "    '''\n",
        "    \n",
        "    m = len(y)\n",
        "    \n",
        "    predictions = X.dot(y_pred)\n",
        "    cost = (1 / 2 * m) * np.sum(np.square(predictions - y))\n",
        "\n",
        "    return cost"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FcXqsVNpDbKC"
      },
      "source": [
        "## Gradients\n",
        "\n",
        "\\begin{equation}\n",
        "y_{pred_0}: = y_{pred_0} -\\alpha . (1/m .\\sum_{i=1}^{m}(h(y_{pred}^{(i)} - y^{(i)}).X_0^{(i)})\n",
        "\\end{equation}\n",
        "\\begin{equation}\n",
        "y_{pred_1}: = y_{pred_1} -\\alpha . (1/m .\\sum_{i=1}^{m}(h(y_{pred}^{(i)} - y^{(i)}).X_0^{(i)})\n",
        "\\end{equation}\n",
        ".\n",
        "\n",
        ".\n",
        "\n",
        ".\n",
        "\n",
        ".\n",
        "\n",
        "\\begin{equation}\n",
        "y_{pred_j}: = y_{pred_j} -\\alpha . (1/m .\\sum_{i=1}^{m}(h(y_{pred}^{(i)} - y^{(i)}).X_0^{(i)})\n",
        "\\end{equation}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fwxBFXP88NBW"
      },
      "source": [
        "def gradient_descent(X, y, y_pred, learning_rate=0.01, iterations=100):\n",
        "    '''\n",
        "    X = Matrix of X with added bias units\n",
        "    y = Vector of Y\n",
        "    y_pred = Vector of y_preds np.random.randn(j, 1)\n",
        "    learning_rate \n",
        "    iterations = no of iterations\n",
        "    \n",
        "    Returns the final y_pred vector and array of cost history over no of iterations\n",
        "    '''\n",
        "\n",
        "    m = len(y)\n",
        "    cost_history = np.zeros(iterations)\n",
        "    y_pred_history = np.zeros((iterations, 2))\n",
        "    \n",
        "    for it in range(iterations):    \n",
        "        prediction = np.dot(X, y_pred)\n",
        "        y_pred = y_pred - (1 / m) * learning_rate * (X.T.dot((prediction - y)))\n",
        "        y_pred_history[it,:] = y_pred.T\n",
        "        cost_history[it]  = cal_cost(y_pred, X, y)\n",
        "        \n",
        "    return y_pred, cost_history, y_pred_history    "
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1iSohSB2EtK1"
      },
      "source": [
        "Let's do 1000 iterations with a learning rate of 0.01. \n",
        "We will start with a random prediction."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "18AX7hrU8bv5",
        "outputId": "bdb90588-c463-446c-ab81-bac3302aca8d"
      },
      "source": [
        "lr = 0.01\n",
        "n_iter = 1000\n",
        "\n",
        "y_pred = np.random.randn(2,1)\n",
        "X_b = np.c_[np.ones((len(X), 1)), X]\n",
        "y_pred, cost_history, y_pred_history = gradient_descent(X_b, y, y_pred, lr, n_iter)\n",
        "\n",
        "print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "print('Final error: {:0.3f}'.format(cost_history[-1]))"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "y_pred[0]: 3.903\n",
            "y_pred[1]: 3.046\n",
            "Final error: 4180.375\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m7fao2MaE216"
      },
      "source": [
        "Plotting the error vs Number of iterations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 514
        },
        "id": "DrkrAAbk8hIs",
        "outputId": "9c3e5c57-cdbc-4f12-ab25-705a351b000a"
      },
      "source": [
        "fig, ax = plt.subplots(figsize=(12,8))\n",
        "\n",
        "ax.set_ylabel('Error')\n",
        "ax.set_xlabel('Number of iterations')\n",
        "\n",
        "ax.plot(range(n_iter), cost_history, 'b.')"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f426c96d3d0>]"
            ]
          },
          "metadata": {},
          "execution_count": 8
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 864x576 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuYAAAHgCAYAAADpKKjTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuLElEQVR4nO3df7xddX3n+9eHE+KvVsOPDNoEm7Rm7KQ6Ip4iu/ahp8aB4PgweLUtXnuTVq6pVVvstBfJ3DvDXOk0amcGyowyjYAmrQ+RAZU8erU0PbKL0zn8OJEKAjKk+INwQTIkgredckz43D/W98jOycnvc/Zae+3X8/E4j7XWd62993ez2fA+3/P5fldkJpIkSZLqdULdHZAkSZJkMJckSZIawWAuSZIkNYDBXJIkSWoAg7kkSZLUAAZzSZIkqQEW1N2Bpjj11FNz2bJldXdDkiRJLbd9+/b/kZmLZ7YbzItly5YxOTlZdzckSZLUchHxndnaLWWRJEmSGsBgLkmSJDWAwVySJElqAIO5JEmS1AAGc0mSJKkBDOaSJElSAxjMJUmSpAYwmEuSJEkNYDCXJEmSGsBgLkmSJDWAwVySJElqAIO5JEmS1AAGc0mSJKkBDOaSJElSAxjMJUmSpAYwmEuSJEkNYDCv0cQEbNxYbSVJkjTcFtTdgWE1MQGrVsHUFCxcCOPj0OnU3StJkiTVxRHzmnS7VSjft6/adrt190iSJEl1MpjXZGysGikfGam2Y2N190iSJEl1spSlJp1OVb7S7Vah3DIWSZKk4WYwr1GnYyCXJElSxVIWSZIkqQEM5pIkSVIDGMwlSZKkBjCYS5IkSQ1gMJckSZIawGAuSZIkNYDBXJIkSWoAg7kkSZLUAAZzSZIkqQEM5jWbmICNG6utJEmShteCujswzCYmYNUqmJqChQthfBw6nbp7JUmSpDo4Yl6jbrcK5fv2Vdtut+4eSZIkqS4G8xqNjVUj5SMj1XZsrO4eSZIkqS6WstSo06nKV7rdKpRbxiJJkjS8DOY163QM5JIkSbKURZIkSWoEg7kkSZLUAPMWzCPi2oh4PCK+Mcu5342IjIhTy3FExJURsSMi7o6IM3uuXRcRD5afdT3tr4mIe8pjroyIKO0nR8S2cv22iDhpvt6jJEmSNFfmc8T808DqmY0RcTpwDvDdnubzgBXlZz1wVbn2ZOBS4LXAWcClPUH7KuA9PY+bfq1LgPHMXAGMl2NJkiSp0eYtmGfmrcDuWU5dDlwMZE/bGmBLVm4DFkXES4BzgW2ZuTsz9wDbgNXl3Asz87bMTGALcH7Pc20u+5t72iVJkqTG6muNeUSsAR7JzK/POLUEeLjneGdpO1T7zlnaAU7LzEfL/mPAaYfoz/qImIyIyV27dh3t25EkSZLmTN+CeUQ8H/iXwL/u12uW0fQ8xPlNmTmamaOLFy/uV7ckSZKkA/RzxPyngeXA1yPi28BS4GsR8WLgEeD0nmuXlrZDtS+dpR3ge6XUhbJ9fM7fiSRJkjTH+hbMM/OezPxHmbksM5dRlZ+cmZmPAVuBtWV1lrOBJ0s5ys3AORFxUpn0eQ5wczn3VEScXVZjWQvcVF5qKzC9esu6nnZJkiSpseZzucTPAhPAyyNiZ0RceIjLvwQ8BOwAPgm8DyAzdwOXAXeWnw+XNso1V5fH/C3w5dL+EeCfRcSDwJvKcaNNTMDGjdVWkiRJwymqMmyNjo7m5ORk3193YgJWrYKpKVi4EMbHodPpezckSZLUJxGxPTNHZ7Z758+adbtVKN+3r9p2u3X3SJIkSXUwmNdsbKwaKR8ZqbZjY3X3SJIkSXVYUHcHhl2nU5WvdLtVKLeMRZIkaTgZzBug0zGQS5IkDTtLWSRJkqQGMJhLkiRJDWAwlyRJkhrAYC5JkiQ1gMFckiRJagCDuSRJktQABnNJkiSpAQzmkiRJUgMYzCVJkqQGMJg3wMQEbNxYbSVJkjScFtTdgWE3MQGrVsHUFCxcCOPj0OnU3StJkiT1myPmNet2q1C+b1+17Xbr7pEkSZLqYDCv2dhYNVI+MlJtx8bq7pEkSZLqYClLzTqdqnyl261CuWUskiRJw8lg3gCdjoFckiRp2FnKIkmSJDWAwVySJElqAIO5JEmS1AAGc0mSJKkBDOaSJElSAxjMJUmSpAYwmEuSJEkNYDCXJEmSGsBg3hATE7BxY7WVJEnS8PHOnw0wMQGrVsHUFCxcCOPj3glUkiRp2Dhi3gDdbhXK9+2rtt1u3T2SJElSvxnMG2BsrBopHxmptmNjdfdIkiRJ/WYpSwN0OlX5SrdbhXLLWCRJkoaPwbwhOh0DuSRJ0jCzlEWSJElqAIO5JEmS1AAGc0mSJKkBDOaSJElSAxjMJUmSpAYwmEuSJEkNYDCXJEmSGsBg3hATE7BxY7WVJEnS8PEGQw0wMQGrVsHUFCxcWN0F1JsNSZIkDRdHzBug261C+b591bbbrbtHkiRJ6jeDeQOMjVUj5SMj1XZsrO4eSZIkqd8sZWmATqcqX+l2q1BuGYskSdLwmbcR84i4NiIej4hv9LT9YUR8MyLujogvRMSinnMbImJHRDwQEef2tK8ubTsi4pKe9uURcXtp/1xELCztzynHO8r5ZfP1HudSpwMbNhjKJUmShtV8lrJ8Glg9o20b8IrM/KfAfwc2AETESuAC4GfLYz4RESMRMQJ8HDgPWAm8s1wL8FHg8sx8GbAHuLC0XwjsKe2Xl+skSZKkRpu3YJ6ZtwK7Z7T9RWbuLYe3AUvL/hrgusx8OjO/BewAzio/OzLzocycAq4D1kREAG8EbiiP3wyc3/Ncm8v+DcCqcr0kSZLUWHVO/nw38OWyvwR4uOfcztJ2sPZTgO/3hPzp9v2eq5x/slwvSZIkNVYtwTwi/k9gL/CZOl6/px/rI2IyIiZ37dpVZ1ckSZI05PoezCPi14C3AO/KzCzNjwCn91y2tLQdrP0JYFFELJjRvt9zlfMvKtcfIDM3ZeZoZo4uXrz4ON+ZJEmSdOz6GswjYjVwMfDWzPz7nlNbgQvKiirLgRXAHcCdwIqyAstCqgmiW0ugvwV4R3n8OuCmnudaV/bfAXyl5xcASZIkqZHmbR3ziPgsMAacGhE7gUupVmF5DrCtzMe8LTPfm5n3RsT1wH1UJS7vz8x95Xk+ANwMjADXZua95SU+BFwXEb8P3AVcU9qvAf4kInZQTT69YL7eoyRJkjRXwsHkyujoaE5OTtbah4kJbzIkSZLUdhGxPTNHZ7Z758+GmJiAVatgagoWLqzuBGo4lyRJGh51LpeoHt1uFcr37au23W7dPZIkSVI/GcwbYmysGikfGam2Y2N190iSJEn9ZClLQ3Q6VfmKNeaSJEnDyWDeIJ2OgVySJGlYWcoiSZIkNYDBXJIkSWoAg7kkSZLUAAZzSZIkqQEM5pIkSVIDGMwlSZKkBjCYN8zEBGzcWG0lSZI0PFzHvEEmJmDVKpiaqu7+OT7uuuaSJEnDwhHzBul2q1C+b1+17Xbr7pEkSZL6xWDeIGNj1Uj5yEi1HRuru0eSJEnqF0tZGqTTqcpXut0qlFvGIkmSNDwM5g3T6RjIJUmShpGlLJIkSVIDGMwlSZKkBjCYS5IkSQ1gMJckSZIawGAuSZIkNYDBXJIkSWoAg3nDTEzAxo3VVpIkScPDdcwbZGICVq2Cqanqzp/j465pLkmSNCwcMW+QbrcK5fv2Vdtut+4eSZIkqV8M5g0yNlaNlI+MVNuxsbp7JEmSpH6xlKVBOp2qfKXbrUK5ZSySJEnDw2DeMJ2OgVySJGkYWcoiSZIkNYDBXJIkSWoAg7kkSZLUAAZzSZIkqQEM5pIkSVIDGMwbaGICNm6stpIkSRoOLpfYMBMTsGpVdefPhQurdc1dPlGSJKn9HDFvmG63CuX79lXbbrfuHkmSJKkfDOYNMzZWjZSPjFTbsbG6eyRJkqR+sJSlYTqdqnyl261CuWUskiRJw8Fg3kCdjoFckiRp2FjKIkmSJDWAwVySJElqAIO5JEmS1AAGc0mSJKkBDOaSJElSAxjMG2hiAjZurLaSJEkaDvMWzCPi2oh4PCK+0dN2ckRsi4gHy/ak0h4RcWVE7IiIuyPizJ7HrCvXPxgR63raXxMR95THXBkRcajXGBQTE7BqFfyrf1VtDeeSJEnDYT5HzD8NrJ7RdgkwnpkrgPFyDHAesKL8rAeugipkA5cCrwXOAi7tCdpXAe/pedzqw7zGQOh2YWoK9u2rtt1u3T2SJElSP8xbMM/MW4HdM5rXAJvL/mbg/J72LVm5DVgUES8BzgW2ZebuzNwDbANWl3MvzMzbMjOBLTOea7bXGAhjY7BwIYyMVNuxsbp7JEmSpH7o950/T8vMR8v+Y8BpZX8J8HDPdTtL26Had87SfqjXGAidDoyPVyPlY2PeAVSSJGlY9DuY/0hmZkRkna8REeupSmd46UtfOp9dOSqdjoFckiRp2PR7VZbvlTIUyvbx0v4IcHrPdUtL26Hal87SfqjXOEBmbsrM0cwcXbx48TG/KUmSJOl49TuYbwWmV1ZZB9zU0762rM5yNvBkKUe5GTgnIk4qkz7PAW4u556KiLPLaixrZzzXbK8hSZIkNda8lbJExGeBMeDUiNhJtbrKR4DrI+JC4DvAL5fLvwS8GdgB/D3w6wCZuTsiLgPuLNd9ODOnJ5S+j2rll+cBXy4/HOI1JEmSpMaKalETjY6O5uTkZN3dkCRJUstFxPbMHJ3Z7p0/G8q7f0qSJA2X2lZl0cFN3/1zaqpay3x83FVaJEmS2s4R8wby7p+SJEnDx2DeQN79U5IkafhYytJA3v1TkiRp+BjMG8q7f0qSJA0XS1kkSZKkBjCYS5IkSQ1gMJckSZIawGAuSZIkNYDBvMG8+6ckSdLwcFWWhvLun5IkScPFEfOG8u6fkiRJw8Vg3lDe/VOSJGm4WMrSUN79U5IkabgYzBvMu39KkiQND0tZJEmSpAYwmEuSJEkNYDCXJEmSGsBgLkmSJDWAwbzBvPOnJEnS8HBVlobyzp+SJEnDxRHzhvLOn5IkScPFYN5Q3vlTkiRpuFjK0lDe+VOSJGm4GMwbzDt/SpIkDQ9LWSRJkqQGMJhLkiRJDWAwlyRJkhrAYN5w3mRIkiRpODj5s8G8yZAkSdLwcMS8wbzJkCRJ0vAwmDeYNxmSJEkaHpayNJg3GZIkSRoeBvOG8yZDkiRJw8FSFkmSJKkBDOaSJElSAxjMJUmSpAYwmA8AbzIkSZLUfk7+bDhvMiRJkjQcHDFvOG8yJEmSNBwM5g3nTYYkSZKGg6UsDedNhiRJkoaDwXwAeJMhSZKk9rOURZIkSWqAWoJ5RPxORNwbEd+IiM9GxHMjYnlE3B4ROyLicxGxsFz7nHK8o5xf1vM8G0r7AxFxbk/76tK2IyIuqeEtSpIkSUel78E8IpYAvw2MZuYrgBHgAuCjwOWZ+TJgD3BheciFwJ7Sfnm5johYWR73s8Bq4BMRMRIRI8DHgfOAlcA7y7UDy3XMJUmS2q+uGvMFwPMi4ofA84FHgTcC/2s5vxn4N8BVwJqyD3AD8J8iIkr7dZn5NPCtiNgBnFWu25GZDwFExHXl2vvm+T3NC9cxlyRJGg6HHTGPiBMi4ufn6gUz8xHg3wHfpQrkTwLbge9n5t5y2U5gSdlfAjxcHru3XH9Kb/uMxxysfSC5jrkkSdJwOGwwz8xnqEpD5kREnEQ1gr0c+AngBVSlKH0XEesjYjIiJnft2lVHFw7LdcwlSZKGw5HWmI9HxNtLCcnxehPwrczclZk/BD4PvA5YFBHTpTVLgUfK/iPA6QDl/IuAJ3rbZzzmYO0HyMxNmTmamaOLFy+eg7c296bXMb/sMstYJEmS2uxIg/lvAP8FmIqIpyLiBxHx1DG+5neBsyPi+SXor6Kq/74FeEe5Zh1wU9nfWo4p57+SmVnaLyirtiwHVgB3AHcCK8oqLwupJohuPca+NkKnAxs2GMolSZLa7Igmf2bmj8/VC2bm7RFxA/A1YC9wF7AJ+H+A6yLi90vbNeUh1wB/UiZ37qYK2mTmvRFxPVWo3wu8PzP3AUTEB4CbqVZ8uTYz752r/kuSJEnzIarB5yO4MOKtwOvLYTcz/2zeelWD0dHRnJycrLsbkiRJarmI2J6ZozPbj6iUJSI+AlxENTp9H3BRRGyc2y7qUFzLXJIkqd2OdB3zNwNnlBVaiIjNVOUmG+arY3qWa5lLkiS139Hc+XNRz/6L5rgfOgTXMpckSWq/Ix0x/wPgroi4BQiqWvNL5q1X2s/0WubTI+auZS5JktQ+hw3mEXEC8AxwNvBzpflDmfnYfHZMz5pey7zbrUK5ZSySJEntc9hgnpnPRMTFmXk9A74e+CDrdAzkkiRJbXakNeZ/GRG/FxGnR8TJ0z/z2jNJkiRpiBxpjfmvlO37e9oS+Km57Y4kSZI0nI60xvySzPxcH/qjg5iYsMZckiSpzY60xvz/AAzmNXEdc0mSpPazxnwAuI65JElS+1ljPgBcx1ySJKn9jiiYZ+by+e6IDs51zCVJktrvkKUsEXFxz/4vzTj3B/PVKR2o04ENGwzlkiRJbXW4GvMLevY3zDi3eo77IkmSJA2twwXzOMj+bMeSJEmSjtHhgnkeZH+2Y82ziQnYuLHaSpIkqV0ON/nzVRHxFNXo+PPKPuX4ufPaM+3HtcwlSZLa7ZAj5pk5kpkvzMwfz8wFZX/6+MR+dVKuZS5JktR2R3qDIdVsei3zkRHXMpckSWqjI73BkGrmWuaSJEntZjAfIJ2OgVySJKmtLGWRJEmSGsBgPmBcMlGSJKmdLGUZIC6ZKEmS1F6OmA8Ql0yUJElqL4P5AHHJREmSpPaylGWAuGSiJElSexnMB4xLJkqSJLWTpSySJElSAxjMB4zLJUqSJLWTpSwDxOUSJUmS2ssR8wHicomSJEntZTAfIC6XKEmS1F6WsgwQl0uUJElqL4P5gHG5REmSpHaylGUAuTKLJElS+zhiPmBcmUWSJKmdHDEfMK7MIkmS1E4G8wHjyiySJEntZCnLgHFlFkmSpHYymA8gV2aRJElqH0tZBpCrskiSJLWPI+YDxlVZJEmS2skR8wHjqiySJEntZDAfMK7KIkmS1E61BPOIWBQRN0TENyPi/ojoRMTJEbEtIh4s25PKtRERV0bEjoi4OyLO7HmedeX6ByNiXU/7ayLinvKYKyMi6nif82F6VZbLLrOMRZIkqU3qGjH/I+DPM/NngFcB9wOXAOOZuQIYL8cA5wErys964CqAiDgZuBR4LXAWcOl0mC/XvKfncav78J76ptOBDRsM5ZIkSW3S92AeES8CXg9cA5CZU5n5fWANsLlcthk4v+yvAbZk5TZgUUS8BDgX2JaZuzNzD7ANWF3OvTAzb8vMBLb0PFdruDKLJElSu9SxKstyYBfwqYh4FbAduAg4LTMfLdc8BpxW9pcAD/c8fmdpO1T7zlnaW8OVWSRJktqnjlKWBcCZwFWZ+Wrg73i2bAWAMtKd892RiFgfEZMRMblr1675frk548oskiRJ7VNHMN8J7MzM28vxDVRB/XulDIWyfbycfwQ4vefxS0vbodqXztJ+gMzclJmjmTm6ePHi43pT/eTKLJIkSe3T92CemY8BD0fEy0vTKuA+YCswvbLKOuCmsr8VWFtWZzkbeLKUvNwMnBMRJ5VJn+cAN5dzT0XE2WU1lrU9z9UKrswiSZLUPnXd+fO3gM9ExELgIeDXqX5JuD4iLgS+A/xyufZLwJuBHcDfl2vJzN0RcRlwZ7nuw5m5u+y/D/g08Dzgy+WnVTodA7kkSVKbRFXOrdHR0ZycnKy7G0dlYqKqLx8bM6RLkiQNiojYnpmjM9vrGjHXcXJlFkmSpHap6wZDOk6uzCJJktQuBvMB5coskiRJ7WIpy4CaXpnFGnNJkqR2MJgPMFdmkSRJag9LWQbYxARs3FhtJUmSNNgcMR9QrsoiSZLULo6YDyhXZZEkSWoXg/mAclUWSZKkdrGUZUC5KoskSVK7GMwH2HQYny5jMZxLkiQNLoP5AHMCqCRJUntYYz7AnAAqSZLUHgbzAeYEUEmSpPawlGWAOQFUkiSpPRwxlyRJkhrAEfMB5uRPSZKk9nDEfIA5+VOSJKk9DOYDzMmfkiRJ7WEpywBz8qckSVJ7GMwHnHf/lCRJageD+YBzAqgkSVI7WGM+4JwAKkmS1A4G8wHnBFBJkqR2sJRlwDkBVJIkqR0M5i3gBFBJkqTBZzBvASeASpIkDT5rzFvACaCSJEmDz2DeAk4AlSRJGnyWsrRApwNXXAE33ghvf7tlLJIkSYPIYN4CExPwwQ9WZSxf/Sq88pWGc0mSpEFjKUsLWGMuSZI0+AzmLWCNuSRJ0uCzlKUFvMmQJEnS4DOYt4Q3GZIkSRpsBvOW8CZDkiRJg80a85ZwAqgkSdJgM5i3hBNAJUmSBpulLC3hTYYkSZIGm8G8JbzJkCRJ0mCzlKUlrDGXJEkabAbzlrDGXJIkabBZytIS0zcZ2rKl7p5IkiTpWDhi3jKbN8MnP1mtaT4xUXdvJEmSdKQM5i1inbkkSdLgMpi3iHXmkiRJg6u2YB4RIxFxV0T8WTleHhG3R8SOiPhcRCws7c8pxzvK+WU9z7GhtD8QEef2tK8ubTsi4pK+v7maTNeZv+c9sG5d3b2RJEnS0ahzxPwi4P6e448Cl2fmy4A9wIWl/UJgT2m/vFxHRKwELgB+FlgNfKKE/RHg48B5wErgneXaoWGduSRJ0uCpJZhHxFLgnwNXl+MA3gjcUC7ZDJxf9teUY8r5VeX6NcB1mfl0Zn4L2AGcVX52ZOZDmTkFXFeuHQrWmUuSJA2mukbMrwAuBp4px6cA38/MveV4J7Ck7C8BHgYo558s1/+ofcZjDtZ+gIhYHxGTETG5a9eu43xLzTA2VtWYR1Rb68wlSZIGQ9+DeUS8BXg8M7f3+7VnysxNmTmamaOLFy+uuztzJmL/rSRJkpqvjhHz1wFvjYhvU5WZvBH4I2BRREzf8Ggp8EjZfwQ4HaCcfxHwRG/7jMccrH0odLuwdy9kVltLWSRJkgZD34N5Zm7IzKWZuYxq8uZXMvNdwC3AO8pl64Cbyv7Wckw5/5XMzNJ+QVm1ZTmwArgDuBNYUVZ5WVheY2sf3lojuGSiJEnSYFpw+Ev65kPAdRHx+8BdwDWl/RrgTyJiB7CbKmiTmfdGxPXAfcBe4P2ZuQ8gIj4A3AyMANdm5r19fSc1ml4yccuWunsiSZKkoxHV4LNGR0dzcnKy7m7MiYmJaqnEqalq1Hx8vArskiRJql9EbM/M0Znt3vmzhVwyUZIkafAYzFtous78hBOqlVlOOaXuHkmSJOlwDOYt1OnAFVdUE0CfeQY++EHvACpJktR0BvOWeuKJKpQ/84zlLJIkSYPAYN5SLpsoSZI0WJq0XKLm0HQ5y403wtvf7qoskiRJTWcwb6mJiaq2fGoKvvpVeOUrDeeSJElNZilLS7lkoiRJ0mAxmLeUSyZKkiQNFoN5S7lkoiRJ0mAxmLeYSyZKkiQNDoN5i42NVSPmEdXWJRMlSZKay2DechH7byVJktRMBvMW63Zh717IrLaWskiSJDWXwbzFXJlFkiRpcBjMW8yVWSRJkgaHwbzleldm+Yd/gC1b6u6RJEmSZmMwb7nplVmgqjX/1KccNZckSWoig3nLdTrw7nc/uyqLk0AlSZKayWA+BNauhRNPdD1zSZKkJjOYDwnXM5ckSWo2g/kQ6F3PfGrKCaCSJElNZDAfAk4AlSRJaj6D+RBwAqgkSVLzGcyHhBNAJUmSms1gPkScACpJktRcBvMh4QRQSZKkZjOYDwkngEqSJDWbwXxIzJwA6qi5JElSsxjMh8j0BFBw1FySJKlpDOZDxGUTJUmSmstgPmRcNlGSJKmZDOZDyGUTJUmSmsdgPmRcNlGSJKmZDOZDxmUTJUmSmslgPmSmJ4BO++EPnQAqSZLUBAbzIfTqVz+7/8wzcMop9fVFkiRJFYP5EHriCTihfPIRcNdd9fZHkiRJBvOhNDYGCxZU+5nwyU/Cpk21dkmSJGnoGcyH0Mw683374AMfcBKoJElSnQzmQ2rt2mdHzcG7gEqSJNXNYD6kOh34F//i2eNMJ4FKkiTVyWA+xBYtchKoJElSUxjMh5iTQCVJkprDYD7EnAQqSZLUHH0P5hFxekTcEhH3RcS9EXFRaT85IrZFxINle1Jpj4i4MiJ2RMTdEXFmz3OtK9c/GBHretpfExH3lMdcGRHR7/c5KJwEKkmS1Ax1jJjvBX43M1cCZwPvj4iVwCXAeGauAMbLMcB5wIrysx64CqogD1wKvBY4C7h0OsyXa97T87jVfXhfA2m2SaDf/35t3ZEkSRpafQ/mmfloZn6t7P8AuB9YAqwBNpfLNgPnl/01wJas3AYsioiXAOcC2zJzd2buAbYBq8u5F2bmbZmZwJae59IsFi2qJn9Ou/xyy1kkSZL6rdYa84hYBrwauB04LTMfLaceA04r+0uAh3setrO0Hap95yztOoixMRgZefbYchZJkqT+qy2YR8SPATcCH8zMp3rPlZHu7EMf1kfEZERM7tq1a75frrEsZ5EkSapfLcE8Ik6kCuWfyczPl+bvlTIUyvbx0v4IcHrPw5eWtkO1L52l/QCZuSkzRzNzdPHixcf3pgac5SySJEn1qmNVlgCuAe7PzP/Qc2orML2yyjrgpp72tWV1lrOBJ0vJy83AORFxUpn0eQ5wczn3VEScXV5rbc9z6SBmK2fZsqW27kiSJA2dOkbMXwf8b8AbI+Jvys+bgY8A/ywiHgTeVI4BvgQ8BOwAPgm8DyAzdwOXAXeWnw+XNso1V5fH/C3w5X68sUHW6cDHP/5sOM+Ea65x1FySJKlfoirn1ujoaE5OTtbdjdq97W3wxS8+e3z++fCFL9TVG0mSpPaJiO2ZOTqz3Tt/aj8vfvH+x1u3OmouSZLUDwZz7WftWjih59+KZ56Bj32svv5IkiQNC4O59tPpwFvfun/bTTfBpk319EeSJGlYGMx1gIsv3n+Flkx43/ssaZEkSZpPBnMdoNOBT3xi/3XN9+2zpEWSJGk+Gcw1q/XrYc2a/ducCCpJkjR/DOY6qIsvdiKoJElSvxjMdVBOBJUkSeofg7kOyYmgkiRJ/WEw1yE5EVSSJKk/DOY6rNkmglrSIkmSNLcM5joiMyeCZsJ732s4lyRJmisGcx2R2SaCGs4lSZLmjsFcR+zii+HEE/dvM5xLkiTNDYO5jlinA3/1V7By5f7thnNJkqTjZzDXUel04OqrHTmXJEmaawZzHbVDjZz/xm/Ahz5UT78kSZIGmcFcx2R65PyEWf4N+tjH4Fd/tf99kiRJGmQGcx2zTgeuumr2cP6Zz8Ab3uAdQiVJko6UwVzHZf16+K//FV7/+gPP3XorvO51lrZIkiQdCYO5jtt0zfm73nXgucyqtGX5cieGSpIkHYrBXHPmT/+0Wut8Nt/+djUx9IwzLG+RJEmajcFcc+qjH4U//mOImP38178OP//z1p9LkiTNZDDXnFu/Hv76r2evO592661VQH/JS+BtbzOkS5IkGcw1L6brzv/bf6vKVw7mscfgi180pEuSJBnMNa86Hbjrrqq85Sd/8tDXzgzpb3gD/OZvGtQlSdJwMJirL9avryaAHklAhyqk33or/Of//GxQX77cEXVJktRekZl196ERRkdHc3Jysu5uDI1Nm+CKK+B734Pdu4/+8S9+cfWzZ0810fSlL4WVK2Ht2mqUXpIkqakiYntmjh7QbjCvGMzrc7whfaYVK2DBAnjOc54N7osWwcKFcOGF1ei9JElSXQzmh2Ewb4bpkL5nTxXU5+NfzyVLquA+Hdh7w/vTTx8Y6J9+Gl7+8mqNdkfjJUnS8TKYH4bBvHkmJmDLFrjvPvjOd+AHP5ibEfXj8eIXw3Ofe/AQfyT7048z8EuSNJwM5odhMB8M0yPq//N/Pht4n366miw6yHoD/+EC/dGE/0Nd+4IXwEUXWdojSVK/GcwPw2A+2CYm4GMfgwceeHY0ujeY7t0LO3fW3ctmOvlkeOEL5+avAHP5OCf0SpLaymB+GAbz9tu0Ca65BqamjjwonngiPPhg3T3XwSb01vVLw7Fe6y8bkiQwmB+WwVwHM7PWfS4C3969Bv5h169fNpr0V5CjufaMM5x/Iam9DOaHYTBXvx0s8PcjjD322ODX5Ws4zLxnQRN+aaj7cU3u21y/J+fCqK0M5odhMNewma7Lv+uu5v2Puw0TeiXNnd65ME35paEJj7Nvx9e3Ou9vcrBgvqD/XZHUBJ0OfOELdffi4A43obeJ/5E/1LX+siEdu927618uV+10xx3Vtil/lTGYS2qkpv/icCzq+GVjEH+hacI9CyQNjxtvNJhL0tBp4y8b82W2exY04ZeGJjyuyX2by/fkXBj1y9vfXncPnmUwlyQ1zvr1zRnBUn1mmwvThF8amvI4+za4NeYHYzCXJEmN5F+ZNGxOqLsDkiRJkgzmkiRJUiMYzCVJkqQGMJhLkiRJDdDaYB4RqyPigYjYERGX1N0fSZIk6VBaGcwjYgT4OHAesBJ4Z0SsrLdXkiRJ0sG1MpgDZwE7MvOhzJwCrgPW1NwnSZIk6aDaGsyXAA/3HO8sbfuJiPURMRkRk7t27epb5yRJkqSZ2hrMj0hmbsrM0cwcXbx4cd3dkSRJ0hBrazB/BDi953hpaZMkSZIaqa3B/E5gRUQsj4iFwAXA1pr7JEmSJB3Ugro7MB8yc29EfAC4GRgBrs3Me2vuliRJknRQrQzmAJn5JeBLdfdDkiRJOhJtLWWRJEmSBorBXJIkSWoAg7kkSZLUAJGZdfehESJiF/CdGl76VOB/1PC66i8/5+Hg5zwc/JyHg5/zcKjrc/7JzDzgJjoG85pFxGRmjtbdD80vP+fh4Oc8HPych4Of83Bo2udsKYskSZLUAAZzSZIkqQEM5vXbVHcH1Bd+zsPBz3k4+DkPBz/n4dCoz9kac0mSJKkBHDGXJEmSGsBgXqOIWB0RD0TEjoi4pO7+6NhFxOkRcUtE3BcR90bERaX95IjYFhEPlu1JpT0i4sry2d8dEWfW+w50pCJiJCLuiog/K8fLI+L28ll+LiIWlvbnlOMd5fyyWjuuIxYRiyLihoj4ZkTcHxEdv8vtExG/U/57/Y2I+GxEPNfv8+CLiGsj4vGI+EZP21F/fyNiXbn+wYhY16/+G8xrEhEjwMeB84CVwDsjYmW9vdJx2Av8bmauBM4G3l8+z0uA8cxcAYyXY6g+9xXlZz1wVf+7rGN0EXB/z/FHgcsz82XAHuDC0n4hsKe0X16u02D4I+DPM/NngFdRfd5+l1skIpYAvw2MZuYrgBHgAvw+t8GngdUz2o7q+xsRJwOXAq8FzgIunQ7z881gXp+zgB2Z+VBmTgHXAWtq7pOOUWY+mplfK/s/oPof+RKqz3RzuWwzcH7ZXwNsycptwKKIeEl/e62jFRFLgX8OXF2OA3gjcEO5ZOZnPP3Z3wCsKterwSLiRcDrgWsAMnMqM7+P3+U2WgA8LyIWAM8HHsXv88DLzFuB3TOaj/b7ey6wLTN3Z+YeYBsHhv15YTCvzxLg4Z7jnaVNA678ifPVwO3AaZn5aDn1GHBa2ffzH0xXABcDz5TjU4DvZ+bectz7Of7oMy7nnyzXq9mWA7uAT5WSpasj4gX4XW6VzHwE+HfAd6kC+ZPAdvw+t9XRfn9r+14bzKU5FBE/BtwIfDAzn+o9l9USSC6DNKAi4i3A45m5ve6+aF4tAM4ErsrMVwN/x7N/9gb8LrdBKUtYQ/WL2E8AL6BPI6KqV9O/vwbz+jwCnN5zvLS0aUBFxIlUofwzmfn50vy96T9rl+3jpd3Pf/C8DnhrRHybqvTsjVS1yIvKn8Jh/8/xR59xOf8i4Il+dljHZCewMzNvL8c3UAV1v8vt8ibgW5m5KzN/CHye6jvu97mdjvb7W9v32mBenzuBFWUG+EKqSSdba+6TjlGpNbwGuD8z/0PPqa3A9GzudcBNPe1ry4zws4Ene/7MpgbKzA2ZuTQzl1F9X7+Sme8CbgHeUS6b+RlPf/bvKNc3dpRGlcx8DHg4Il5emlYB9+F3uW2+C5wdEc8v//2e/pz9PrfT0X5/bwbOiYiTyl9Xzilt884bDNUoIt5MVbM6Alybmf+23h7pWEXELwBfBe7h2frjf0lVZ3498FLgO8AvZ+bu8j+C/0T1p9O/B349Myf73nEdk4gYA34vM98SET9FNYJ+MnAX8KuZ+XREPBf4E6r5BruBCzLzoZq6rKMQEWdQTfBdCDwE/DrVQJbf5RaJiP8b+BWqVbXuAv53qjpiv88DLCI+C4wBpwLfo1pd5Ysc5fc3It5N9f9xgH+bmZ/qS/8N5pIkSVL9LGWRJEmSGsBgLkmSJDWAwVySJElqAIO5JEmS1AAGc0mSJKkBDOaS1GcRkRHx73uOfy8i/s0cPfenI+Idh7/yuF/nlyLi/oi4ZUb7T0TEDWX/jLIs7Fy95qKIeN9sryVJbWAwl6T+exr4XyLi1Lo70qvnjodH4kLgPZn5i72Nmfn/Zub0LwZnAEcVzA/Th0XAj4L5jNeSpIFnMJek/tsLbAJ+Z+aJmSPeEfH/le1YRPxVRNwUEQ9FxEci4l0RcUdE3BMRP93zNG+KiMmI+O8R8Zby+JGI+MOIuDMi7o6I3+h53q9GxFaqOx/O7M87y/N/IyI+Wtr+NfALwDUR8Yczrl9Wrl0IfBj4lYj4m4j4lYh4QURcW/p8V0SsKY/5tYjYGhFfAcYj4sciYjwivlZee015+o8AP12e7w+nX6s8x3Mj4lPl+rsi4hd7nvvzEfHnEfFgRHys55/Hp0tf74mIAz4LSeq3oxkdkSTNnY8Dd08HxSP0KuCfUN158CHg6sw8KyIuAn4L+GC5bhlwFvDTwC0R8TJgLdXtpn8uIp4D/HVE/EW5/kzgFZn5rd4Xi4ifAD4KvAbYA/xFRJyfmR+OiDdS3f101rtcZuZUCfCjmfmB8nx/QHUr83dHxCLgjoj4y54+/NNyN74FwNsy86nyV4Xbyi8Ol5R+nlGeb1nPS76/etl8ZUT8TOnrPy7nzqC6Y+PTwAMR8R+BfwQsycxXlOdadIh/7pLUF46YS1INMvMpYAvw20fxsDsz89HMfBr4W2A6WN9DFcanXZ+Zz2Tmg1QB/meAc4C1EfE3wO3AKcCKcv0dM0N58XNANzN3ZeZe4DPA64+ivzOdA1xS+tAFnkt1i2yAbZm5u+wH8AcRcTfwl1S3ST/tMM/9C8CfAmTmN6luuz0dzMcz88nM/Aeqvwr8JNU/l5+KiP8YEauBp47jfUnSnHDEXJLqcwXwNeBTPW17KYMmEXECsLDn3NM9+8/0HD/D/v89zxmvk1Rh97cy8+beExExBvzdsXT+GATw9sx8YEYfXjujD+8CFgOvycwfRsS3qUL8ser957YPWJCZeyLiVcC5wHuBXwbefRyvIUnHzRFzSapJGSG+nmoi5bRvU5WOALwVOPEYnvqXIuKEUnf+U8ADwM3Ab0bEiQAR8Y8j4gWHeZ47gDdExKkRMQK8E/iro+jHD4Af7zm+GfitiIjSh1cf5HEvAh4vofwXqUa4Z3u+Xl+lCvSUEpaXUr3vWZUSmRMy80bg/6IqpZGkWhnMJale/x7oXZ3lk1Rh+OtAh2Mbzf4uVaj+MvDeUsJxNVUZx9fKhMk/5jB/Nc3MR6nqum8Bvg5sz8ybjqIftwArpyd/ApdR/aJxd0TcW45n8xlgNCLuoaqN/2bpzxNUtfHfmDnpFPgEcEJ5zOeAXyslPwezBOiWspo/BTYcxfuSpHkRmTP/4ilJkiSp3xwxlyRJkhrAYC5JkiQ1gMFckiRJagCDuSRJktQABnNJkiSpAQzmkiRJUgMYzCVJkqQGMJhLkiRJDfD/AyD4ld4bv/phAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IG5tWAy-FCaW"
      },
      "source": [
        "Zooming in..."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 500
        },
        "id": "WZ7BoFHy8kTk",
        "outputId": "c71a82cd-d931-406e-d7a6-cee4a049a1d3"
      },
      "source": [
        "fig,ax = plt.subplots(figsize=(10,8))\n",
        "ax.plot(range(200), cost_history[:200], 'b.')"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f426c91a400>]"
            ]
          },
          "metadata": {},
          "execution_count": 9
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 720x576 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmkAAAHSCAYAAAC3lFz5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAl10lEQVR4nO3db7Ck1X0f+O+PweA/iQRCE1YLTMAxcQonmxjfkrkVxzXlUdCg9RolcblQXMXEokRSlhIrzpYDcWmhTFJEya7lUJFJ4SAJXIqQotglKpGC0Viz9ouLrEGWhJAsM5YsMRQSBLCUWiWMgbMvnueKZnT/9r19++nuz6fqVneffrr7PPTM3C/nPL9zqrUWAACG5YxpdwAAgG8npAEADJCQBgAwQEIaAMAACWkAAAMkpAEADNCZ0+7AbnvlK1/ZLr744ml3AwBgUw8++OB/a63tX+u5uQtpF198cY4fPz7tbgAAbKqqvrzec6Y7AQAGSEgDABggIQ0AYICENACAARLSAAAGSEgDABggIQ0AYICENACAARLSAAAGSEgDABggIQ0AYICENACAARLSAAAGSEgDABggIQ0AYICENACAARLStmllJbn11u4WAGBSzpx2B2bJykpy6FBy6lRy1lnJ0aPJ8vK0ewUAzCMjadtw7FgX0J5/vrs9dmzaPQIA5pWQtg0HD3YjaPv2dbcHD067RwDAvDLduQ3Ly90U57FjXUAz1QkATIqQtk3Ly8IZADB5pjsBAAZISAMAGCAhDQBggIQ0AIABEtIAAAZISAMAGCAhDQBggIQ0AIABEtIAAAZISAMAGCAhbQdWVpJbb+1uAQB2k707x7Sykhw6lJw6lZx1Vrfxuj09AYDdYiRtTMeOdQHt+ee722PHpt0jAGCeCGljOniwG0Hbt6+7PXhw2j0CAOaJ6c4xLS93U5zHjnUBzVQnALCbhLQdWF4WzgCAyTDdCQAwQJuGtKp6V1U9UVWfXeO5f1JVrape2T+uqrqtqk5U1Weq6vKRY49U1SP9z5GR9h+qqof619xWVdW3v6Kq7u+Pv7+qzt2dUwYAGL6tjKS9J8nh0xur6qIkVyb5ykjzVUku7X+uT3J7f+wrktyU5IeTvDrJTSOh6/Ykbxp53epn3ZDkaGvt0iRH+8cAAAth05DWWvudJE+v8dQ7kvxCkjbSdnWSu1vngSTnVNWrkrw2yf2ttadba88kuT/J4f65l7XWHmittSR3J3n9yHvd1d+/a6QdAGDujXVNWlVdneSx1tqnT3vqgiSPjjw+2bdt1H5yjfYkOb+19nh//6tJzh+nrwAAs2jb1Z1V9d1J/lm6qc490VprVdXWe76qrk83vZoDBw7sVbcAACZmnJG0v5DkkiSfrqo/TnJhkk9W1f+S5LEkF40ce2HftlH7hWu0J8nX+unQ9LdPrNeh1todrbWl1trS/v37xzglAIBh2XZIa6091Fr7c621i1trF6ebory8tfbVJPcmubav8rwiydf7Kcv7klxZVef2BQNXJrmvf+4bVXVFX9V5bZIP9R91b5LVKtAjI+0AAHNvK0twvC/JSpLvr6qTVXXdBod/OMkXk5xI8mtJfjZJWmtPJ7klySf6n1/q29If8+/71/xRko/07f8yyd+sqkeSvKZ/DACwEKorqpwfS0tL7fjx43v6mSsrtocCALavqh5srS2t9ZxtoXZoZSU5dCg5darbaP3oUUENANg520Lt0LFjXUB7/vnu9tixafcIAJgHQtoOHTzYjaDt29fdHjw47R4BAPPAdOcOLS93U5yuSQMAdpOQtguWl4UzAGB3me4EABggIQ0AYICENACAARLSAAAGSEgDABggIQ0AYICENACAARLSAAAGSEgDABggIW2Xrawkt97a3QIAjMu2ULtoZSU5dCg5darbbP3oUdtFAQDjMZK2i44d6wLa8893t8eOTbtHAMCsEtJ20cGD3Qjavn3d7cGD0+4RADCrTHfuouXlborz2LEuoJnqBADGJaTtsuVl4QwA2DnTnQAAAySkAQAMkJAGADBAQhoAwAAJaQAAAySkAQAMkJAGADBAQhoAwAAJaRO0spLcemt3CwCwHXYcmJCVleTQoW6j9bPO6raLshMBALBVRtIm5NixLqA9/3x3e+zYtHsEAMwSIW1CDh7sRtD27etuDx6cdo8AgFliunNClpe7Kc5jx7qAZqoTANgOIW2ClpeFMwBgPKY7AQAGSEgDABggIQ0AYICENACAARLSAAAGSEgDABggIQ0AYICEtD1go3UAYLssZjthNloHAMZhJG3CbLQOAIxDSJswG60DAOMw3TlhNloHAMax6UhaVb2rqp6oqs+OtP3rqvqDqvpMVf1mVZ0z8tyNVXWiqr5QVa8daT/ct52oqhtG2i+pqo/37e+vqrP69rP7xyf65y/erZPea8vLyY03CmgAwNZtZbrzPUkOn9Z2f5K/3Fr735L8YZIbk6SqLktyTZIf6F/zq1W1r6r2JXlnkquSXJbkDf2xSfL2JO9orX1fkmeSXNe3X5fkmb79Hf1xAAALYdOQ1lr7nSRPn9b2W6215/qHDyS5sL9/dZJ7WmvPtta+lOREklf3Pydaa19srZ1Kck+Sq6uqkvxYkg/2r78ryetH3uuu/v4HkxzqjwcAmHu7UTjwxiQf6e9fkOTRkedO9m3rtZ+X5E9GAt9q+0veq3/+6/3xAABzb0chrap+MclzSd67O90Zux/XV9Xxqjr+5JNPTrMrAAC7YuyQVlV/L8mPJ/np1lrrmx9LctHIYRf2beu1P5XknKo687T2l7xX//zL++O/TWvtjtbaUmttaf/+/eOeEgDAYIwV0qrqcJJfSPITrbVvjjx1b5Jr+srMS5JcmuT3knwiyaV9JedZ6YoL7u3D3ceS/GT/+iNJPjTyXkf6+z+Z5LdHwiAAwFzbdJ20qnpfkoNJXllVJ5PclK6a8+wk9/fX8j/QWvsHrbWHq+oDST6Xbhr0za215/v3eUuS+5LsS/Ku1trD/Uf80yT3VNU/T/L7Se7s2+9M8utVdSJd4cI1u3C+AAAzoeZtcGppaakdP3582t1Y18qKhW0BgE5VPdhaW1rrOTsO7CGbrQMAW2Xvzj1ks3UAYKuEtD1ks3UAYKtMd+4hm60DAFslpO2x5WXhDADYnOlOAIABEtIAAAZISAMAGCAhDQBggIQ0AIABEtIAAAZISAMAGCAhbUpWVpJbb+1uAQBOZzHbKbDROgCwGSNpU2CjdQBgM0LaFNhoHQDYjOnOKbDROgCwGSFtSmy0DgBsxHQnAMAACWkAAAMkpAEADJCQBgAwQEIaAMAACWkAAAMkpA2AfTwBgNNZJ23K7OMJAKzFSNqU2ccTAFiLkDZl9vEEANZiunPK7OMJAKxFSBsA+3gCAKcz3QkAMEBCGgDAAAlpAAADJKQBAAyQkAYAMEBC2oDYHgoAWGUJjoGwPRQAMMpI2kDYHgoAGCWkDYTtoQCAUaY7B8L2UADAKCFtQGwPBQCsMt0JADBAQhoAwAAJaQAAAySkAQAMkJAGADBAQtpA2SIKABbbpiGtqt5VVU9U1WdH2l5RVfdX1SP97bl9e1XVbVV1oqo+U1WXj7zmSH/8I1V1ZKT9h6rqof41t1VVbfQZi2B1i6i3va27FdQAYPFsZSTtPUkOn9Z2Q5KjrbVLkxztHyfJVUku7X+uT3J70gWuJDcl+eEkr05y00jouj3Jm0Zed3iTz5h7togCADYNaa2130ny9GnNVye5q79/V5LXj7Tf3ToPJDmnql6V5LVJ7m+tPd1aeybJ/UkO98+9rLX2QGutJbn7tPda6zPmni2iAIBxdxw4v7X2eH//q0nO7+9fkOTRkeNO9m0btZ9co32jz5h7togCAHa8LVRrrVVV243OjPsZVXV9uunVHDhwYJJd2TO2iAKAxTZudefX+qnK9LdP9O2PJblo5LgL+7aN2i9co32jz/g2rbU7WmtLrbWl/fv3j3lKAADDMW5IuzfJaoXmkSQfGmm/tq/yvCLJ1/spy/uSXFlV5/YFA1cmua9/7htVdUVf1Xntae+11mcAAMy9Tac7q+p9SQ4meWVVnUxXpfkvk3ygqq5L8uUkP9Uf/uEkr0tyIsk3k/xMkrTWnq6qW5J8oj/ul1prq8UIP5uugvS7knyk/8kGnwEAMPeqK6qcH0tLS+348ePT7gYAwKaq6sHW2tJaz9lxAABggIS0gbM9FAAsph0vwcHkrG4PdepUt6jt0aOW5QCARWEkbcBsDwUAi0tIGzDbQwHA4jLdOWC2hwKAxSWkDZztoQBgMZnuBAAYICENAGCAhDQAgAES0gAABkhImyF2HwCAxaG6c0bYfQAAFouRtBlh9wEAWCxC2oyw+wAALBbTnTPC7gMAsFiEtBli9wEAWBymOwEABkhIAwAYICENAGCAhLQZZFFbAJh/CgdmjEVtAWAxGEmbMRa1BYDFIKTNGIvaAsBiMN05YyxqCwCLQUibQRa1BYD5Z7oTAGCAhDQAgAES0gAABkhIAwAYICFtxtl9AADmk+rOGWb3AQCYX0bSZpjdBwBgfglpM8zuAwAwv0x3zjC7DwDA/BLSZpzdBwBgPpnuBAAYICENAGCAhLQ5Ys00AJgfrkmbE9ZMA4D5YiRtTlgzDQDmi5A2J6yZBgDzxXTnnLBmGgDMFyFtjlgzDQDmh+lOAIABEtIAAAZoRyGtqv5xVT1cVZ+tqvdV1XdW1SVV9fGqOlFV76+qs/pjz+4fn+ifv3jkfW7s279QVa8daT/ct52oqht20lcAgFkydkirqguS/KMkS621v5xkX5Jrkrw9yTtaa9+X5Jkk1/UvuS7JM337O/rjUlWX9a/7gSSHk/xqVe2rqn1J3pnkqiSXJXlDfyybsKgtAMy+nRYOnJnku6rqT5N8d5LHk/xYkr/bP39XkpuT3J7k6v5+knwwyb+tqurb72mtPZvkS1V1Ismr++NOtNa+mCRVdU9/7Od22Oe5ZlFbAJgPY4+ktdYeS/J/J/lKunD29SQPJvmT1tpz/WEnk1zQ378gyaP9a5/rjz9vtP2016zXzgYsagsA82En053nphvZuiTJ/5rke9JNV+65qrq+qo5X1fEnn3xyGl0YDIvaAsB82Ml052uSfKm19mSSVNVvJPnrSc6pqjP70bILkzzWH/9YkouSnKyqM5O8PMlTI+2rRl+zXvtLtNbuSHJHkiwtLbUdnNPMs6gtAMyHnYS0ryS5oqq+O8n/SHIoyfEkH0vyk0nuSXIkyYf64+/tH6/0z/92a61V1b1J/kNV/XK6EblLk/xekkpyaVVdki6cXZMXr3VjAxa1BYDZN3ZIa619vKo+mOSTSZ5L8vvpRrP+S5J7quqf92139i+5M8mv94UBT6cLXWmtPVxVH0hXEPBckje31p5Pkqp6S5L70lWOvqu19vC4/QUAmCXV2nzNDi4tLbXjx49PuxsAAJuqqgdba0trPWfHgTlnzTQAmE02WJ9j1kwDgNllJG2OWTMNAGaXkDbHrJkGALPLdOccs2YaAMwuIW3OWTMNAGaT6U4AgAES0gAABkhIWxDWSwOA2eKatAVgvTQAmD1G0haA9dIAYPYIaQvAemkAMHtMdy4A66UBwOwR0haE9dIAYLaY7gQAGCAhDQBggIS0BWTNNAAYPtekLRhrpgHAbDCStmCsmQYAs0FIWzDWTAOA2WC6c8FYMw0AZoOQtoCsmQYAw2e6EwBggIS0BWYpDgAYLtOdC8pSHAAwbEbSFpSlOABg2IS0BWUpDgAYNtOdC8pSHAAwbELaArMUBwAMl+lOAIABEtJIYjkOABga051YjgMABshIGpbjAIABEtKwHAcADJDpTizHAQADJKSRxHIcADA0pjsBAAZISOMlLMUBAMNgupNvsRQHAAyHkTS+xVIcADAcQhrfYikOABgO0518i6U4AGA4hDRewlIcADAMpjtZl0pPAJgeI2msSaUnAEyXkTTWpNITAKZLSGNNKj0BYLp2FNKq6pyq+mBV/UFVfb6qlqvqFVV1f1U90t+e2x9bVXVbVZ2oqs9U1eUj73OkP/6Rqjoy0v5DVfVQ/5rbqqp20l+2brXS85ZbTHUCwDTsdCTt3yT5r621v5Tkryb5fJIbkhxtrV2a5Gj/OEmuSnJp/3N9ktuTpKpekeSmJD+c5NVJbloNdv0xbxp53eEd9pdtWF5ObrxRQAOAaRg7pFXVy5P8aJI7k6S1dqq19idJrk5yV3/YXUle39+/OsndrfNAknOq6lVJXpvk/tba0621Z5Lcn+Rw/9zLWmsPtNZakrtH3os9pMoTAPbeTqo7L0nyZJJ3V9VfTfJgkp9Lcn5r7fH+mK8mOb+/f0GSR0def7Jv26j95Brt36aqrk83OpcDBw6Mf0Z8G1WeADAdO5nuPDPJ5Ulub639YJL/Ly9ObSZJ+hGwtoPP2JLW2h2ttaXW2tL+/fsn/XELRZUnAEzHTkLaySQnW2sf7x9/MF1o+1o/VZn+9on++ceSXDTy+gv7to3aL1yjnT2kyhMApmPskNZa+2qSR6vq+/umQ0k+l+TeJKsVmkeSfKi/f2+Sa/sqzyuSfL2fFr0vyZVVdW5fMHBlkvv6575RVVf0VZ3XjrwXe0SVJwBMx053HPiHSd5bVWcl+WKSn0kX/D5QVdcl+XKSn+qP/XCS1yU5keSb/bFprT1dVbck+UR/3C+11p7u7/9skvck+a4kH+l/2GP28wSAvVfdZWPzY2lpqR0/fnza3ZhbKyvddWkHDwpuALBTVfVga21prefs3cmWqfQEgL1jWyi2TKUnAOwdIY0tU+kJAHvHdCdbtlrp6Zo0AJg8IY1tUekJAHvDdCdjs6cnAEyOkTTGotITACbLSBpjUekJAJMlpDEWlZ4AMFmmOxmLSk8AmCwjaYxteTm58cbuvgICANhdRtLYEQUEADAZRtLYEQUEADAZQho7ooAAACbDdCc7ooAAACZDSGPHRreKWlkR2ABgNwhp7BpFBACwe1yTxq5RRAAAu0dIY9coIgCA3WO6k12jiAAAdo+RNHaVXQgAYHcYSWPXKSAAgJ0zksauU0AAADsnpLHrFBAAwM6Z7mTXKSAAgJ0T0pgIuxAAwM4IaUyUIgIAGI9r0pgoRQQAMB4hjYlSRAAA4zHdyUQpIgCA8RhJY+LsQgAA22ckjT2hgAAAtsdIGntCAQEAbI+Qxp5QQAAA2yOksSdWCwhuuaW7TVyfBgAbcU0ae2Z1FwLXpwHA5oyksedcnwYAmxPS2HOuTwOAzZnuZM+NLnB73nkvjqSZ8gSAFwlpTMVqIHNtGgCszXQnU+PaNABYn5DG1Lg2DQDWJ6QxNdZOA4D1uSaNqbJ2GgCszUgag+D6NAB4KSGNQXB9GgC81I5DWlXtq6rfr6r/3D++pKo+XlUnqur9VXVW3352//hE//zFI+9xY9/+hap67Uj74b7tRFXdsNO+MlyuTwOAl9qNkbSfS/L5kcdvT/KO1tr3JXkmyXV9+3VJnunb39Efl6q6LMk1SX4gyeEkv9oHv31J3pnkqiSXJXlDfyxzank5ufHG7v6hQ8nb3tbdCmoALKIdhbSqujDJ/57k3/ePK8mPJflgf8hdSV7f37+6f5z++UP98Vcnuae19mxr7UtJTiR5df9zorX2xdbaqST39Mcy51yfBgA7H0n7lSS/kOSF/vF5Sf6ktfZc//hkkgv6+xckeTRJ+ue/3h//rfbTXrNeO3PO9WkAsIMlOKrqx5M80Vp7sKoO7lqPxuvL9UmuT5IDBw5MsyvsAnt7AsDO1kn760l+oqpel+Q7k7wsyb9Jck5VndmPll2Y5LH++MeSXJTkZFWdmeTlSZ4aaV81+pr12l+itXZHkjuSZGlpqe3gnBgIe3sCsOjGnu5srd3YWruwtXZxugv/f7u19tNJPpbkJ/vDjiT5UH//3v5x+ud/u7XW+vZr+urPS5JcmuT3knwiyaV9tehZ/WfcO25/mT2uTQNgkU1inbR/muTnq+pEumvO7uzb70xyXt/+80luSJLW2sNJPpDkc0n+a5I3t9ae70fi3pLkvnTVox/oj2VBnH5t2nnnWZYDgMVR3WDW/FhaWmrHjx+fdjfYJSsrL16b9ta3mvoEYL5U1YOttaW1nrPjAIO2unbaU0+Z+gRgsQhpzATLcgCwaHZS3Ql7xrIcACwaIY2ZYVkOABaJ6U5mimU5AFgUQhozxbIcACwK053MlNOvTbMsBwDzykgaM8eyHAAsAiGNmTU69blvX/KVr5j2BGB+CGnMrNWpzze9KalKfu3XuspPQQ2AeSCkMdOWl5MDB5LnnjPtCcB8EdKYeSo+AZhHqjuZeSo+AZhHRtKYCyo+AZg3QhpzRcUnAPNCSGOuqPgEYF4IacwdFZ8AzAMhjbmk4hOAWae6k7mk4hOAWWckjbml4hOAWSakMfdUfAIwi4Q05p6KTwBmkZDGQlir4vPuuxUTADBcCgdYGKvTnqdOddOe7353F9oUEwAwREbSWBir05633JK88Y3WUQNg2IQ0Fspqxee111pHDYBhM93JQrKOGgBDZySNhbXWOmrPPpvcfLMRNQCmT0hj4a0WFJxxRvLCC8lHP2qJDgCmT0hj4a1Ofb7mNS8GNUt0ADBt1Vqbdh921dLSUjt+/Pi0u8EMWlnpRtBWl+ioskQHAJNVVQ+21pbWes5IGvQs0QHAkAhpMGKtJTrs9wnANAhpsAb7fQIwbUIarMN+nwBMk8VsYQP2+wRgWoykwQbWKyaw6C0AkyakwSZOLyaw6C0Ae0FIgy2y6C0Ae8litrBNFr0FYLdYzBZ2kevUANgLRtJgB1ZH1Z59tpv+POOM5Oyzk1/5leSpp7rqUCNrAKzHSBpMyFrXqT37bPKWtyRve5vCAgDGJ6TBDi0vd9OcZ5/dXaN2xhnd9KcFcAHYCYvZwi5YHVE7diw577zkrW+1AC4AOyOkwS5ZXn4xgP2Vv9IFtq98pdv3c7Sw4OabBTUANjf2dGdVXVRVH6uqz1XVw1X1c337K6rq/qp6pL89t2+vqrqtqk5U1Weq6vKR9zrSH/9IVR0Zaf+hqnqof81tVVU7OVnYK5stgHvHHaZAAdjYTq5Jey7JP2mtXZbkiiRvrqrLktyQ5Ghr7dIkR/vHSXJVkkv7n+uT3J50oS7JTUl+OMmrk9y0Guz6Y9408rrDO+gv7DmFBQCMa+yQ1lp7vLX2yf7+f0/y+SQXJLk6yV39YXcleX1//+okd7fOA0nOqapXJXltkvtba0+31p5Jcn+Sw/1zL2utPdC6dULuHnkvmBkbFRZYWw2A9ezKNWlVdXGSH0zy8STnt9Ye75/6apLz+/sXJHl05GUn+7aN2k+u0Q4zZ63CgtW11T760eR3f9faagC81I5DWlX9mST/KclbW2vfGL1srLXWqmriq+VW1fXpplBz4MCBSX8cjOX0woKbb+4C2ugU6AsvqAIFoLOjddKq6jvSBbT3ttZ+o2/+Wj9Vmf72ib79sSQXjbz8wr5to/YL12j/Nq21O1prS621pf379+/klGBPmAIFYDM7qe6sJHcm+Xxr7ZdHnro3yWqF5pEkHxppv7av8rwiydf7adH7klxZVef2BQNXJrmvf+4bVXVF/1nXjrwXzLzRPUDf+c4usKkCBWDV2Ht3VtWPJPndJA8leaFv/mfprkv7QJIDSb6c5Kdaa0/3QevfpqvQ/GaSn2mtHe/f6439a5PkX7TW3t23LyV5T5LvSvKRJP+wbdJhe3cyq1ZWXjoFesYZ3SjbCy90t298Y7ekh2lQgPmx0d6dNliHAVndsP3UqaSqC2gv9P8LVJV853cqMACYJxuFNDsOwICsVQX6P/9n0lr3o8AAYHHYYB0GZnW3guuv70LY3//7CgwAFpHpTpgBKytrr7F2xhldgDMFCjCbTHfCjNvOGmsCG8B8MJIGM2i9AgMVoQCzRXUnzKHTp0BVhALMHiEN5tzpgW21IjQxugYwZEIaLJCVleTuu5N3vzt57jmjawBDJqTBAhpndC3pXiO4AewNIQ0W3FZG177jO7rb554zLQqwV4Q0IMnGo2tV3e3oY9OiAJMlpAHf5vTRtX37umB26pSiA4C9IqQB61odXTt4sHu83aKDxHVsAOMS0oBt2U7RgevYAMYnpAFj26joYLPr2M47z/VsABsR0oAdW2uHg42uY3v++bU3gRfcAF4kpAG7ajvXsSWCG8B6hDRg4k4faXv22RcD2RlnbD+4KUoAFoGQBuyp0cC2OlK2neC2UVFCIrgB80NIA6Zuu8FtraKE9XZFSAQ3YDYJacAgrRfc1itKGCe4ud4NGDIhDZgZGxUlbDe4/emfbl6okBiFA6ZHSANm2k6DW7J2ocKZZ744CnfWWUIcsPeENGDubCW4jY6krVWoMBrmNgtxm02lrrYJc8B2CGnAQjg9uG1WqDAawtZa322rU6mbhbm1+iPMAYmQBpDk2wsVTg9P64W4zaZSt1LQsN1r5IzSwWIQ0gC2YL0Qt9lU6mZhbrvXyG1nlG69MLfZ88IeDIOQBrBDG02lbhTmxr1Gbr22rU657kbY204AFPpgPEIawB5ZK8xt9xq57YzSrdps5G67YW87AXCcqVuhEDpCGsAAbXaN3DhTruNOya4aJwBud+p2FkPhOO8jSLIVQhrAHNjKlOs4U7I7DU3jTt1upW0IoXDc99lqle+k2qb53kLq1glpAHzLVsPeVn85b3fqdtZC4bhtW6ny3Y3/TpMMnNO47nGvA+dmnzfpwCmkATBR25m6nbVQuN332U6V7yTbpv3eoyF1CN/LOJ+3uhvJ0aOTC2pCGgBzZS9D4XbbtlPlO48jaTu97nGttmkG3H37kltuSW68MRMhpAHAHtpqle80rxub1Hvv5LpHI2mnPSekAQC7aSfXPbombeQ5IQ0AYDo2Cmln7HVnAADYnJAGADBAQhoAwAAJaQAAAySkAQAMkJAGADBAQhoAwAAJaQAAAySkAQAMkJAGADBAgw9pVXW4qr5QVSeq6oZp9wcAYC8MOqRV1b4k70xyVZLLkryhqi6bbq8AACZv0CEtyauTnGitfbG1dirJPUmunnKfAAAmbugh7YIkj448Ptm3vURVXV9Vx6vq+JNPPrlnnQMAmJShh7Qtaa3d0Vpbaq0t7d+/f9rdAQDYsaGHtMeSXDTy+MK+DQBgrlVrbdp9WFdVnZnkD5McShfOPpHk77bWHt7gNU8m+fKEu/bKJP9twp8xZM5/cc9/kc89cf7Of3HPf5HPPZns+f/51tqa04BnTugDd0Vr7bmqekuS+5LsS/KujQJa/5qJz3dW1fHW2tKkP2eonP/inv8in3vi/J3/4p7/Ip97Mr3zH3RIS5LW2oeTfHja/QAA2EtDvyYNAGAhCWnjuWPaHZgy57+4FvncE+fv/BfXIp97MqXzH3ThAADAojKSBgAwQELaNi3Shu9VdVFVfayqPldVD1fVz/XtN1fVY1X1qf7nddPu66RU1R9X1UP9eR7v215RVfdX1SP97bnT7uckVNX3j3zHn6qqb1TVW+f5+6+qd1XVE1X12ZG2Nb/v6tzW/1vwmaq6fHo93x3rnP+/rqo/6M/xN6vqnL794qr6HyN/Dv7d1Dq+C9Y593X/rFfVjf13/4Wqeu10er171jn/94+c+x9X1af69nn77tf7XTf9v/utNT9b/Em3DMgfJfneJGcl+XSSy6bdrwme76uSXN7f/7Pp1qy7LMnNSf7Pafdvj/4b/HGSV57W9q+S3NDfvyHJ26fdzz3477AvyVeT/Pl5/v6T/GiSy5N8drPvO8nrknwkSSW5IsnHp93/CZ3/lUnO7O+/feT8Lx49btZ/1jn3Nf+s9/8OfjrJ2Uku6X8v7Jv2Oez2+Z/2/P+T5P+a0+9+vd91U/+7byRtexZqw/fW2uOttU/29/97ks9njb1TF9DVSe7q79+V5PXT68qeOZTkj1prk14oeqpaa7+T5OnTmtf7vq9OcnfrPJDknKp61Z50dELWOv/W2m+11p7rHz6QbueXubPOd7+eq5Pc01p7trX2pSQn0v1+mFkbnX9VVZKfSvK+Pe3UHtngd93U/+4LaduzpQ3f51FVXZzkB5N8vG96Sz/M+655ne7rtSS/VVUPVtX1fdv5rbXH+/tfTXL+dLq2p67JS/+BXpTvP1n/+17Efw/emG4EYdUlVfX7VfX/VtXfmFanJmytP+uL9t3/jSRfa609MtI2l9/9ab/rpv53X0hjU1X1Z5L8pyRvba19I8ntSf5Ckr+W5PF0w+Dz6kdaa5cnuSrJm6vqR0efbN3Y91yXSFfVWUl+Isl/7JsW6ft/iUX4vtdTVb+Y5Lkk7+2bHk9yoLX2g0l+Psl/qKqXTat/E7Kwf9ZP84a89H/S5vK7X+N33bdM6+++kLY9C7fhe1V9R7o/tO9trf1GkrTWvtZae7619kKSX8uMD/NvpLX2WH/7RJLfTHeuX1sd2u5vn5heD/fEVUk+2Vr7WrJY339vve97Yf49qKq/l+THk/x0/8sq/VTfU/39B9Ndl/UXp9bJCdjgz/oiffdnJvnbSd6/2jaP3/1av+sygL/7Qtr2fCLJpVV1ST+6cE2Se6fcp4npr0O4M8nnW2u/PNI+Ovf+t5J89vTXzoOq+p6q+rOr99NdQP3ZdN/5kf6wI0k+NJ0e7pmX/F/0onz/I9b7vu9Ncm1f6XVFkq+PTI3Mjao6nOQXkvxEa+2bI+37q2pff/97k1ya5IvT6eVkbPBn/d4k11TV2VV1Sbpz/7297t8eeU2SP2itnVxtmLfvfr3fdRnC3/1pV1XM2k+6qo4/TPd/Dr847f5M+Fx/JN3w7meSfKr/eV2SX0/yUN9+b5JXTbuvEzr/701XwfXpJA+vft9JzktyNMkjST6a5BXT7usE/xt8T5Knkrx8pG1uv/90YfTxJH+a7jqT69b7vtNVdr2z/7fgoSRL0+7/hM7/RLrrb1b/Dfh3/bF/p/978akkn0zyf0y7/xM493X/rCf5xf67/0KSq6bd/0mcf9/+niT/4LRj5+27X+933dT/7ttxAABggEx3AgAMkJAGADBAQhoAwAAJaQAAAySkAQAMkJAGADBAQhoAwAAJaQAAA/T/AzrDIEhJdnCzAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JYhOp3fjnh2G"
      },
      "source": [
        "# Stochastic Gradient Descent"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In Batch Gradient Descent we were considering all the examples for every step of Gradient Descent. But what if our dataset is very huge. Deep learning models crave for data. The more the data the more chances of a model to be good. Suppose our dataset has 5 million examples, then just to take one step the model will have to calculate the gradients of all the 5 million examples. This does not seem an efficient way. To tackle this problem we have Stochastic Gradient Descent. In Stochastic Gradient Descent (SGD), we consider just one example at a time to take a single step."
      ],
      "metadata": {
        "id": "10N2dcwWUctJ"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aVwD7Cqw8m1d"
      },
      "source": [
        "def stocashtic_gradient_descent(X, y, y_pred, learning_rate=0.01, iterations=10):\n",
        "    '''\n",
        "    X = Matrix of X with added bias units\n",
        "    y = Vector of Y\n",
        "    y_pred = Vector of y_pred np.random.randn(j,1)\n",
        "    learning_rate \n",
        "    iterations = no of iterations\n",
        "    \n",
        "    Returns the final y_pred vector and array of cost history over no of iterations\n",
        "    '''\n",
        "\n",
        "    m = len(y)\n",
        "    cost_history = np.zeros(iterations)\n",
        "    \n",
        "    for it in range(iterations):\n",
        "        cost = 0.0\n",
        "        \n",
        "        for i in range(m):\n",
        "            rand_ind = np.random.randint(0,m)\n",
        "            X_i = X[rand_ind, :].reshape(1, X.shape[1])\n",
        "            y_i = y[rand_ind].reshape(1,1)\n",
        "            prediction = np.dot(X_i, y_pred)\n",
        "\n",
        "            y_pred = y_pred - (1 / m) * learning_rate *(X_i.T.dot((prediction - y_i)))\n",
        "            cost += cal_cost(y_pred, X_i, y_i)\n",
        "\n",
        "        cost_history[it]  = cost\n",
        "        \n",
        "    return y_pred, cost_history"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Yk6pfB5c8tPz",
        "outputId": "17a9f8e9-2779-43f3-d180-be53ddbb359b"
      },
      "source": [
        "lr = 0.5\n",
        "n_iter = 50\n",
        "y_pred = np.random.randn(2, 1)\n",
        "X_b = np.c_[np.ones((len(X), 1)), X]\n",
        "y_pred, cost_history = stocashtic_gradient_descent(X_b, y, y_pred, lr, n_iter)\n",
        "\n",
        "print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "print('Final error: {:0.3f}'.format(cost_history[-1]))"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "y_pred[0]: 4.121\n",
            "y_pred[1]: 2.755\n",
            "Final error: 41.313\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 514
        },
        "id": "YiJUgS7o8u2e",
        "outputId": "e24817c4-4291-46d7-f2bd-81124e4d98d2"
      },
      "source": [
        "fig, ax = plt.subplots(figsize=(10,8))\n",
        "\n",
        "ax.set_ylabel('Error')\n",
        "ax.set_xlabel('Number of iterations')\n",
        "y_pred = np.random.randn(2,1)\n",
        "\n",
        "ax.plot(range(n_iter), cost_history, 'b.')"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f426c917460>]"
            ]
          },
          "metadata": {},
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 720x576 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmQAAAHgCAYAAAAL2HHvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAeDElEQVR4nO3dfbBnd10f8PcnGyKOUgJkzWASXJA4lLEScKXZytiFVAvokLQCwtASNdNoGy1WLQanU6u2BOooPox1mhIkVFRSlCbDUCEuWbWd5WFDIOGxRAolaSArT8E6kib59I/fuXpZ9uHezT33+7v3vl4zO7/zdM/vc3/n7rnv+/1+zznV3QEAYJzTRhcAALDTCWQAAIMJZAAAgwlkAACDCWQAAIMJZAAAg50+uoAH46yzzuo9e/aMLgMA4KRuvvnmP+vu3cdat6UD2Z49e3L48OHRZQAAnFRVfeJ463RZAgAMJpABAAwmkAEADCaQAQAMJpABAAwmkAEADCaQAQAMJpABAAwmkAEADCaQAQAMJpABAAwmkAEADCaQAQAMJpABAAwmkAEADCaQAQAMJpCdwKFDyVVXLV4BAOZy+ugCltWhQ8lFFyX33puccUZy4ECyb9/oqgCA7UgL2XEcPLgIY/ffv3g9eHB0RQDAdiWQHcf+/YuWsV27Fq/794+uCADYrnRZHse+fYtuyoMHF2FMdyUAMBeB7AT27RPEAID5zdplWVUfr6rbquq9VXV4WvbIqrqxqj46vT5iWl5V9atVdXtV3VpVT5mzNgCAZbEZY8ie3t0XdPfeaf7KJAe6+/wkB6b5JHlWkvOnf5cn+Y1NqA0AYLgRg/ovTnLtNH1tkktWLX9dL7wjyZlV9egB9QEAbKq5A1kneVtV3VxVl0/Lzu7uu6bpTyU5e5o+J8knV33tHdMyAIBtbe5B/U/r7jur6uuS3FhVH169sru7qno9O5yC3eVJ8pjHPGbjKgUAGGTWFrLuvnN6vTvJm5I8NcmnV7oip9e7p83vTHLeqi8/d1p29D6v7u693b139+7dc5YPALApZgtkVfU1VfWwlekk35Xk/UluSHLptNmlSa6fpm9I8uLpassLk3xhVdcmAMC2NWeX5dlJ3lRVK+/z2939B1X17iTXVdVlST6R5PnT9m9J8uwktyf5iyQ/MGNtAABLY7ZA1t0fS/KkYyz/TJKLjrG8k1wxVz0AAMvKsywBAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAabPZBV1a6quqWq3jzNP7aq3llVt1fVG6rqjGn5V03zt0/r98xdGwDAMtiMFrKXJPnQqvlXJnlVdz8+yeeSXDYtvyzJ56blr5q2AwDY9mYNZFV1bpLvTvLqab6SPCPJG6dNrk1yyTR98TSfaf1F0/YAANva3C1kv5zkpUkemOYfleTz3X3fNH9HknOm6XOSfDJJpvVfmLb/MlV1eVUdrqrDR44cmbF0AIDNMVsgq6rvSXJ3d9+8kfvt7qu7e2937929e/dG7hoAYIjTZ9z3tyd5TlU9O8lDk/yNJL+S5MyqOn1qBTs3yZ3T9ncmOS/JHVV1epKHJ/nMjPUBACyF2VrIuvtl3X1ud+9J8oIkb+/uFyW5Kclzp80uTXL9NH3DNJ9p/du7u+eqDwBgWYy4D9lPJfnxqro9izFi10zLr0nyqGn5jye5ckBtAACbbs4uy7/S3QeTHJymP5bkqcfY5i+TPG8z6gEAWCbu1A8AMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADDYbIGsqh5aVe+qqvdV1Qeq6men5Y+tqndW1e1V9YaqOmNa/lXT/O3T+j1z1QYAsEzmbCH7UpJndPeTklyQ5JlVdWGSVyZ5VXc/Psnnklw2bX9Zks9Ny181bQcAsO3NFsh64c+n2YdM/zrJM5K8cVp+bZJLpumLp/lM6y+qqpqrPgCAZTHrGLKq2lVV701yd5Ibk/xpks93933TJnckOWeaPifJJ5NkWv+FJI+asz4AgGUwayDr7vu7+4Ik5yZ5apInPNh9VtXlVXW4qg4fOXLkwe4OAGC4TbnKsrs/n+SmJPuSnFlVp0+rzk1y5zR9Z5LzkmRa//AknznGvq7u7r3dvXf37t1zlw4AMLs5r7LcXVVnTtNfneQ7k3woi2D23GmzS5NcP03fMM1nWv/27u656gMAWBann3yTU/boJNdW1a4sgt913f3mqvpgkt+tqn+b5JYk10zbX5PkP1fV7Uk+m+QFM9YGALA0Zgtk3X1rkicfY/nHshhPdvTyv0zyvLnqAQBYVu7UDwAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADCYQAYAMJhABgAwmEAGADDYSQNZVZ1WVX9nM4oBANiJThrIuvuBJL++CbUAAOxIa+2yPFBV31tVNWs1AAA70FoD2Q8l+S9J7q2qe6rqi1V1z4x1AQDsGKevZaPuftjchQAA7FRrCmRJUlXPSfId0+zB7n7zPCUBAOwsa+qyrKpXJHlJkg9O/15SVVfNWRgAwE6x1hayZye5YLriMlV1bZJbkrxsrsIAAHaK9dwY9sxV0w/f4DoAAHastbaQvTzJLVV1U5LKYizZlbNVBQCwg5w0kFXVaUkeSHJhkm+bFv9Ud39qzsIAAHaKkway7n6gql7a3dcluWETagIA2FHWOobsD6vqJ6vqvKp65Mq/WSsDANgh1jqG7Pum1ytWLeskj9vYcgAAdp61jiG7srvfsAn1AADsOCftspzuPfYvN6EWAIAdyRgyAIDBjCEDABhsTYGsux87dyEAADvVCbssq+qlq6afd9S6l89VFADATnKyMWQvWDV99IPEn7nBtQAA7EgnC2R1nOljzQMAcApOFsj6ONPHmgcA4BScbFD/k6rqnixaw756ms40/9BZKwMA2CFOGMi6e9dmFQIAsFOt9cawAADMRCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGGy2QFZV51XVTVX1war6QFW9ZFr+yKq6sao+Or0+YlpeVfWrVXV7Vd1aVU+ZqzYAgGUyZwvZfUl+orufmOTCJFdU1ROTXJnkQHefn+TANJ8kz0py/vTv8iS/MWNtAABLY7ZA1t13dfd7pukvJvlQknOSXJzk2mmza5NcMk1fnOR1vfCOJGdW1aPnqg8AYFlsyhiyqtqT5MlJ3pnk7O6+a1r1qSRnT9PnJPnkqi+7Y1oGALCtzR7Iquprk/xekh/r7ntWr+vuTtLr3N/lVXW4qg4fOXJkAysFABhj1kBWVQ/JIoy9vrt/f1r86ZWuyOn17mn5nUnOW/Xl507Lvkx3X93de7t77+7du+crHgBgk8x5lWUluSbJh7r7l1atuiHJpdP0pUmuX7X8xdPVlhcm+cKqrk0AgG3r9Bn3/e1J/nGS26rqvdOyn07yiiTXVdVlST6R5PnTurckeXaS25P8RZIfmLE2AIClMVsg6+7/nqSOs/qiY2zfSa6Yqx4AgGXlTv0AAIMJZAAAgwlkAACDCWQAAIMJZAAAgwlkAACDCWQAAIMJZAAAgwlkAACDCWQAAIMJZAAAgwlkAACDCWQAAIMJZAAAgwlkAACDCWQAAIMJZAAAgwlkAACDCWQAAIMJZAAAgwlkAACDCWQb5NCh5KqrFq8AAOtx+ugCtoNDh5KLLkruvTc544zkwIFk377RVQEAW4UWsg1w8OAijN1//+L14MHRFQEAW4lAtgH271+0jO3atXjdv390RQDAVqLLcgPs27fopjx4cBHGdFcCAOshkG2QffsEMQDg1OiyBAAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGGy2QFZVr6mqu6vq/auWPbKqbqyqj06vj5iWV1X9alXdXlW3VtVT5qoLAGDZzNlC9tokzzxq2ZVJDnT3+UkOTPNJ8qwk50//Lk/yGzPWBQCwVGYLZN39x0k+e9Tii5NcO01fm+SSVctf1wvvSHJmVT16rtoAAJbJZo8hO7u775qmP5Xk7Gn6nCSfXLXdHdMyAIBtb9ig/u7uJL3er6uqy6vqcFUdPnLkyAyVAQBsrs0OZJ9e6YqcXu+elt+Z5LxV2507LfsK3X11d+/t7r27d++etVgAgM2w2YHshiSXTtOXJrl+1fIXT1dbXpjkC6u6NgEAtrXT59pxVf1Okv1JzqqqO5L8TJJXJLmuqi5L8okkz582f0uSZye5PclfJPmBueoCAFg2swWy7n7hcVZddIxtO8kVc9UCALDM3KkfAGAwgQwAYDCBDABgMIFsBzl0KLnqqsUrALA8ZhvUz3I5dCi56KLk3nuTM85IDhxI9u0bXRUAkGgh2zEOHlyEsfvvX7wePDi6IgBghUC2pDa6e3H//kXL2K5di9f9+zdmvwDAg6fLcgnN0b24b99iPwcPLsKY7koAWB4C2RI6VvfiRgSoffs2PogdOiTkAcCDJZBtsrUEmJXuxZUWspN1L44KRS4UAICNIZBtorUGmPV0L44MRXO15AHATiOQbaL1BJi1di+ODEXrbckDAI5NINtEcwSYkaHIhQIAsDGqu0fXcMr27t3bhw8fHl3Guswx3svAegBYflV1c3fvPeY6gQwAYH4nCmRuDAsAMJhABgAwmEAGADCYQAYAMJhABgAwmEDGpjh0KLnqqsUrAPDl3BiW2XnmJQCcmBYyZnesxzsBAH9NIGN2K4932rXLMy8B4Fh0WTI7z7wEgBMTyNgU+/atPYh5NicAO41AxlJZzwUAghsA24VAxlI51gUAxwpbrtwEYDsxqJ9jGnXfsLVeAODKTQC2Ey1kfIWRrU9rvQBgJbit1OjKTQC2MoGMr7DWbsO5rOUCAFduAiuMJ2U7EMj4Clul9Wk9V24C25PxpGwXxpDxFVZan37+553clpHngsJfM56U7UILGcek9Wk5aQ2AL7dVWvThZLSQwRaiNWD5bacWzK3wvWjRZ7vQQgZbiNaA5badbmy8lVpjteizHQhksIW4unS5bacbG4++2hoerGX/o+doAhksibWePLQGnNxaP8uNPmGvtQVzK4QdrbHbx1YLJhthK/zRczSBjG1v5MloPcFg5MljO52w1/pZzvGZb6cbG2uN3R5Gn1tG2Qp/9BxNIGNbW+/JaCNbVtbz3iNPHqNP2BsdBtf6Wc71mY+8sfFGf5ZaY7e+rRhMNsJW+KPnaAIZ29p6TkYb3bKynvee4+Sx1l/OWyUMrvX7WetnOfqEvdFhZ3SwZjmN/jlfj438g2IrtvAKZGxr6zkZbXTLynree6NPHuv55TzyhD3HIPi1fpZb8YR9Iju1JWSrGDUsYPTP+chhG+v5o2cZhm0IZGxr6zkZbXTLynpPhBvZYrKeX84jT9hzDYJf62e5nbrk1hOsl+GXz8nMUeOo73t06+Won3PDNtZHIGPbW88v541uWRl1Ilxvq9eoOrfTIPjR1vpZLssvnxOZo8aR3/d6w8ZWCMxrMXrYxhx1zkkgg1W2S8vK6EHj6/mFMnIQ/Hazls9yWX75nMgcNY78vtfbernsgXmtRg7bWI9l+YNPIINtatSg8bl+oSx7CN4qluWXz4nMUeNc3/da/vhYT9jYCoF5rUYO21iPZfmDTyAD1mT07SR2qjluZbHst9yYo8b17HOOgehrDRtbITAn2+9G1stQp0AGrMlWuZ3EdrJVWhtHXyG3kfscPRB9WVprTmQ7dasuE4EMWJOdejuJkbZKa+N67/e3zD8byzAQfRlaa05k9M/lsv8MnSqBDFiz7XLRw1axVVob11rnVmhZ2SoD0ZNxwWTkz+VW+Bk6VQIZwJIa/Qt/rdZa5+iWlbXYKgPRRwaTkT+XW+Fn6FQJZABLbKu0Nq6lzq3S4rcVPvO5gsmyD9bfKj9Dp0IgA2BTbJUWv61gruffLnt34Hb+GRLIANg0W6H1aSuYI5hsle7A7fozJJABwBa00cFkO3cHbgUCGcBJbNfL7GG17dwduBUIZAAnsBXG1cBG2a7dgVvBaaMLAFhmxxpXA7DRBDKAE1gZV7Nrl3E1wHx0WQKcgHE1wGYQyABOwrgaYG66LAEABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAYTyAAABhPIAAAGE8gAAAZbqkBWVc+sqo9U1e1VdeXoegAANsPSBLKq2pXk15M8K8kTk7ywqp44tioAgPktTSBL8tQkt3f3x7r73iS/m+TiwTUBAMxumQLZOUk+uWr+jmnZl6mqy6vqcFUdPnLkyKYVBwAwl9NHF7Be3X11kquTpKqOVNUnZn7Ls5L82czvwalzfJaXY7PcHJ/l5vgsrwdzbL7heCuWKZDdmeS8VfPnTsuOq7t3z1pRkqo63N17534fTo3js7wcm+Xm+Cw3x2d5zXVslqnL8t1Jzq+qx1bVGUlekOSGwTUBAMxuaVrIuvu+qvqRJG9NsivJa7r7A4PLAgCY3dIEsiTp7rckecvoOo5y9egCOCHHZ3k5NsvN8Vlujs/ymuXYVHfPsV8AANZomcaQAQDsSALZCXiU03KpqtdU1d1V9f5Vyx5ZVTdW1Uen10eMrHGnqqrzquqmqvpgVX2gql4yLXd8Bquqh1bVu6rqfdOx+dlp+WOr6p3T+e0N08VUDFJVu6rqlqp68zTv+CyJqvp4Vd1WVe+tqsPTsg0/twlkx+FRTkvptUmeedSyK5Mc6O7zkxyY5tl89yX5ie5+YpILk1wx/X9xfMb7UpJndPeTklyQ5JlVdWGSVyZ5VXc/Psnnklw2rkSSvCTJh1bNOz7L5endfcGq211s+LlNIDs+j3JaMt39x0k+e9Tii5NcO01fm+SSzayJhe6+q7vfM01/MYtfLOfE8RmuF/58mn3I9K+TPCPJG6fljs1AVXVuku9O8uppvuL4LLsNP7cJZMe3pkc5MdzZ3X3XNP2pJGePLIakqvYkeXKSd8bxWQpTd9h7k9yd5MYkf5rk891937SJ89tYv5zkpUkemOYfFcdnmXSSt1XVzVV1+bRsw89tS3XbC3gwururymXDA1XV1yb5vSQ/1t33LP7QX3B8xunu+5NcUFVnJnlTkieMrYgVVfU9Se7u7purav/gcji2p3X3nVX1dUlurKoPr165Uec2LWTHt+5HOTHEp6vq0Ukyvd49uJ4dq6oekkUYe313//602PFZIt39+SQ3JdmX5MyqWvmj3PltnG9P8pyq+ngWQ2OekeRX4vgsje6+c3q9O4s/aJ6aGc5tAtnxeZTT1nBDkkun6UuTXD+wlh1rGvNyTZIPdfcvrVrl+AxWVbunlrFU1Vcn+c4sxvjdlOS502aOzSDd/bLuPre792Txe+bt3f2iOD5Loaq+pqoetjKd5LuSvD8znNvcGPYEqurZWfTtrzzK6d+NrWhnq6rfSbI/yVlJPp3kZ5L81yTXJXlMkk8keX53Hz3wn5lV1dOS/EmS2/LX42B+OotxZI7PQFX1LVkMOt6VxR/h13X3z1XV47JokXlkkluS/KPu/tK4Spm6LH+yu7/H8VkO03F40zR7epLf7u5/V1WPygaf2wQyAIDBdFkCAAwmkAEADCaQAQAMJpABAAwmkAEADCaQARuiqrqqfnHV/E9W1b/ZoH2/tqqee/ItH/T7PK+qPlRVNx21/Our6o3T9AXTLXE26j3PrKp/dqz3AnYOgQzYKF9K8g+r6qzRhay26m7na3FZkn/S3U9fvbC7/093rwTCC5KsK5CdpIYzk/xVIDvqvYAdQiADNsp9Sa5O8i+OXnF0C1dV/fn0ur+q/qiqrq+qj1XVK6rqRVX1rqq6raq+cdVu/l5VHa6q/zk9/2/lodm/UFXvrqpbq+qHVu33T6rqhiQfPEY9L5z2//6qeuW07F8neVqSa6rqF47afs+07RlJfi7J91XVe6vq+6Y7eb9mqvmWqrp4+prvr6obqurtSQ5U1ddW1YGqes/03hdPu39Fkm+c9vcLK+817eOhVfWb0/a3VNXTV+3796vqD6rqo1X171d9Hq+dar2tqr7iWADLycPFgY3060luXQkIa/SkJH8zyWeTfCzJq7v7qVX1kiQ/muTHpu32ZPEMuW9MclNVPT7Ji5N8obu/raq+Ksn/qKq3Tds/Jck3d/f/Wv1mVfX1SV6Z5FuTfC7J26rqkunu9c/I4k7ph49VaHffOwW3vd39I9P+Xp7F425+cHpE0buq6g9X1fAt3f3ZqZXsH0wPXT8ryTumwHjlVOcF0/72rHrLKxZv23+rqp4w1fpN07oLkjw5i5bJj1TVryX5uiTndPc3T/s68wSfO7BEtJABG6a770nyuiT/fB1f9u7uvmt6LMyfJlkJVLdlEcJWXNfdD3T3R7MIbk/I4rlyL66q92bxmKZHJTl/2v5dR4exybclOdjdR7r7viSvT/Id66j3aN+V5MqphoNJHprF41SS5MZVj1OpJC+vqluT/GGSc5KcfZJ9Py3JbyVJd384i0e0rASyA939he7+yyxaAb8hi8/lcVX1a1X1zCT3PIjvC9hEWsiAjfbLSd6T5DdXLbsv0x+AVXVakjNWrVv9fL4HVs0/kC8/Rx39nLfOIuT8aHe/dfWK6ZmA//dUij8FleR7u/sjR9Xwt4+q4UVJdif51u7+f1X18SzC26la/bndn+T07v5cVT0pyd9P8sNJnp/kBx/EewCbRAsZsKGmFqHrshggv+LjWXQRJslzkjzkFHb9vKo6bRpX9rgkH0ny1iT/tKoekiRV9U1V9TUn2c+7kvzdqjqrqnYleWGSP1pHHV9M8rBV829N8qNVVVMNTz7O1z08yd1TGHt6Fi1ax9rfan+SRZDL1FX5mCy+72OaukJP6+7fS/KvsugyBbYAgQyYwy8mWX215X/KIgS9L8m+nFrr1f/OIkz9tyQ/PHXVvTqL7rr3TAPh/2NO0vLf3XdlMW7rpiTvS3Jzd1+/jjpuSvLElUH9SX4+i4B5a1V9YJo/ltcn2VtVt2Ux9u3DUz2fyWLs2/uPvpggyX9Ictr0NW9I8v1T1+7xnJPk4NR9+ltJXraO7wsYqLqP7gUAAGAzaSEDABhMIAMAGEwgAwAYTCADABhMIAMAGEwgAwAYTCADABhMIAMAGOz/AzZ/xMzaKSD9AAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ScckWktynk1o"
      },
      "source": [
        "# Mini Batch Gradient Descent"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We have seen the Batch Gradient Descent. We have also seen the Stochastic Gradient Descent. Batch Gradient Descent can be used for smoother curves. SGD can be used when the dataset is large. Batch Gradient Descent converges directly to minima. SGD converges faster for larger datasets. But, since in SGD we use only one example at a time, we cannot implement the vectorized implementation on it. This can slow down the computations. To tackle this problem, a mixture of Batch Gradient Descent and SGD is used.\n",
        "Neither we use all the dataset all at once nor we use the single example at a time. We use a batch of a fixed number of training examples which is less than the actual dataset and call it a mini-batch. Doing this helps us achieve the advantages of both the former variants we saw."
      ],
      "metadata": {
        "id": "ZTVz-QssUkuE"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4JtxFVL78wEm"
      },
      "source": [
        "def minibatch_gradient_descent(X, y, y_pred, learning_rate=0.01, iterations=10, batch_size=20):\n",
        "    '''\n",
        "    X = Matrix of X without added bias units\n",
        "    y = Vector of Y\n",
        "    y_pred = Vector of y_preds np.random.randn(j, 1)\n",
        "    learning_rate \n",
        "    iterations = no of iterations\n",
        "    \n",
        "    Returns the final theta vector and array of cost history over no of iterations\n",
        "    '''\n",
        "\n",
        "    m = len(y)\n",
        "    cost_history = np.zeros(iterations)\n",
        "    n_batches = int(m / batch_size)\n",
        "    \n",
        "    for it in range(iterations):\n",
        "        cost = 0.0\n",
        "        indices = np.random.permutation(m)\n",
        "        X = X[indices]\n",
        "        y = y[indices]\n",
        "\n",
        "        for i in range(0, m, batch_size):\n",
        "            X_i = X[i: i + batch_size]\n",
        "            y_i = y[i: i + batch_size]\n",
        "            \n",
        "            X_i = np.c_[np.ones(len(X_i)), X_i]\n",
        "            prediction = np.dot(X_i, y_pred)\n",
        "\n",
        "            y_pred = y_pred - (1 / m) * learning_rate * (X_i.T.dot((prediction - y_i)))\n",
        "            cost += cal_cost(y_pred, X_i, y_i)\n",
        "\n",
        "        cost_history[it]  = cost\n",
        "        \n",
        "    return y_pred, cost_history"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SpbsVwA28znL",
        "outputId": "607e8dc1-e6bf-447d-f603-ffc8a7122df6"
      },
      "source": [
        "lr = 0.1\n",
        "n_iter = 200\n",
        "y_pred = np.random.randn(2,1)\n",
        "y_pred, cost_history = minibatch_gradient_descent(X, y, y_pred, lr, n_iter)\n",
        "\n",
        "print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "print('Final error: {:0.3f}'.format(cost_history[-1]))"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "y_pred[0]: 4.148\n",
            "y_pred[1]: 2.827\n",
            "Final error: 807.488\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 514
        },
        "id": "Q_ivOYHT817C",
        "outputId": "f27c8b5f-3fcf-4a7b-bb53-97672cc4ebab"
      },
      "source": [
        "fig, ax = plt.subplots(figsize=(10,8))\n",
        "\n",
        "ax.set_ylabel('Error')\n",
        "ax.set_xlabel('Number of iterations')\n",
        "y_pred = np.random.randn(2,1)\n",
        "\n",
        "ax.plot(range(n_iter), cost_history, 'b.')"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f426c879c40>]"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 720x576 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnAAAAHgCAYAAAAlnVB9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjdElEQVR4nO3df5Ctd10f8PcnN1xw8EdCuM1gEpuAsW1qa8A1ZFtrV2NDkrEGq2IYp4nCGK1goa3VpHYKAx0DWqylg7ShRJJKDdQfwx0HG8I1WzudC2QDMSH8MLcIkjQkV0IJ1pprbr7943kWDsvu3r337tmz393Xa+bMc873POd5Pt89++O93+/znKdaawEAoB+nzLoAAACOjwAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ05ddYFbLVnPvOZ7dxzz511GQAAx3TXXXf9SWtt38r2XRfgzj333CwtLc26DACAY6qqT63WbgoVAKAzAhwAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBbpMdPJjccMOwBACYhlNnXcBOcvBgcsklyZEjyd69yYEDyfz8rKsCAHYaI3CbaHFxCG9Hjw7LxcVZVwQA7EQC3CZaWBhG3vbsGZYLC7OuCADYiUyhbqL5+WHadHFxCG+mTwGAaRDgNtn8vOAGAEyXKVQAgM4IcAAAnRHgAAA6I8ABAHRGgAMA6IwABwDQGQEOAKAzAhwAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0ZmoBrqrOqao7quojVXVfVb1ibH91VT1YVXePtysmXnN9VR2qqo9X1Qsm2i8b2w5V1XUT7edV1fvH9ndU1d5p9QcAYLuY5gjcE0n+WWvtgiQXJ3lZVV0wPvdvW2sXjrd3J8n43FVJ/nqSy5L8SlXtqao9Sd6U5PIkFyR58cR2Xj9u6xuTfC7JS6fYHwCAbWFqAa619lBr7YPj/S8k+WiSs9Z5yZVJbm2tPd5a+6Mkh5JcNN4OtdY+0Vo7kuTWJFdWVSX5riS/Mb7+5iQvnEpnAAC2kS05Bq6qzk3y3CTvH5teXlX3VNVNVXX62HZWkk9PvOyBsW2t9jOS/J/W2hMr2lfb/7VVtVRVS4cPH96MLgEAzMzUA1xVfXWS30zyytbaY0nenOQ5SS5M8lCSN0y7htbaja21udba3L59+6a9OwCAqTp1mhuvqqdkCG9vb639VpK01h6eeP4tSX5nfPhgknMmXn722JY12j+b5LSqOnUchZtcHwBgx5rmWaiV5K1JPtpa+6WJ9mdNrPZ9ST483t+f5KqqempVnZfk/CQfSHJnkvPHM073ZjjRYX9rrSW5I8kPjK+/Jsm7ptUfAIDtYpojcH87yT9Mcm9V3T22/YsMZ5FemKQl+WSSH0+S1tp9VfXOJB/JcAbry1prR5Okql6e5LYke5Lc1Fq7b9zezya5tar+dZIPZQiMAAA7Wg0DWbvH3NxcW1pamnUZAADHVFV3tdbmVra7EgMAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcAAAnRHgAAA6I8ABAHRGgAMA6IwABwDQGQEOAKAzAhwAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcAAAnRHgAAA6I8ABAHRGgAMA6IwABwDQGQEOAKAzAhwAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcAAAnZlagKuqc6rqjqr6SFXdV1WvGNufUVW3V9X94/L0sb2q6o1Vdaiq7qmq501s65px/fur6pqJ9m+tqnvH17yxqmpa/QEA2C6mOQL3RJJ/1lq7IMnFSV5WVRckuS7Jgdba+UkOjI+T5PIk54+3a5O8ORkCX5JXJXl+kouSvGo59I3r/NjE6y6bYn8AALaFqQW41tpDrbUPjve/kOSjSc5KcmWSm8fVbk7ywvH+lUluaYP3JTmtqp6V5AVJbm+tPdpa+1yS25NcNj73ta2197XWWpJbJrYFALBjbckxcFV1bpLnJnl/kjNbaw+NT30myZnj/bOSfHriZQ+Mbeu1P7BK+2r7v7aqlqpq6fDhwyfXGQCAGZt6gKuqr07ym0le2Vp7bPK5ceSsTbuG1tqNrbW51trcvn37pr07AICpmmqAq6qnZAhvb2+t/dbY/PA4/Zlx+cjY/mCScyZefvbYtl772au0AwDsaNM8C7WSvDXJR1trvzTx1P4ky2eSXpPkXRPtV49no16c5PPjVOttSS6tqtPHkxcuTXLb+NxjVXXxuK+rJ7YFALBjnTrFbf/tJP8wyb1VdffY9i+SvC7JO6vqpUk+leRF43PvTnJFkkNJ/izJjyZJa+3RqnptkjvH9V7TWnt0vP+TSd6W5KuS/O54AwDY0Wo4DG33mJuba0tLS7MuAwDgmKrqrtba3Mp2V2IAAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcAAAnRHgAAA6I8ABAHRGgAMA6IwABwDQGQEOAKAzAhwAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMC3BQdPJjccMOwBADYLKfOuoCd6uDB5JJLkiNHkr17kwMHkvn5WVcFAOwERuCmZHFxCG9Hjw7LxcVZVwQA7BQC3JQsLAwjb3v2DMuFhVlXBADsFKZQp2R+fpg2XVwcwpvpUwBgswhwUzQ/L7gBAJvPFCoAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcAAAnRHgAAA6I8ABAHRGgAMA6IwABwDQGQEOAKAzAhwAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM5MLcBV1U1V9UhVfXii7dVV9WBV3T3erph47vqqOlRVH6+qF0y0Xza2Haqq6ybaz6uq94/t76iqvdPqCwDAdjLNEbi3JblslfZ/21q7cLy9O0mq6oIkVyX56+NrfqWq9lTVniRvSnJ5kguSvHhcN0leP27rG5N8LslLp9gXAIBtY2oBrrX2+0ke3eDqVya5tbX2eGvtj5IcSnLReDvUWvtEa+1IkluTXFlVleS7kvzG+Pqbk7xwM+sHANiuZnEM3Mur6p5xivX0se2sJJ+eWOeBsW2t9jOS/J/W2hMr2ldVVddW1VJVLR0+fHiz+gEAMBNbHeDenOQ5SS5M8lCSN2zFTltrN7bW5lprc/v27duKXQIATM2pW7mz1trDy/er6i1Jfmd8+GCScyZWPXtsyxrtn01yWlWdOo7CTa4PALCjbekIXFU9a+Lh9yVZPkN1f5KrquqpVXVekvOTfCDJnUnOH8843ZvhRIf9rbWW5I4kPzC+/pok79qKPgAAzNrURuCq6teTLCR5ZlU9kORVSRaq6sIkLcknk/x4krTW7quqdyb5SJInkrystXZ03M7Lk9yWZE+Sm1pr9427+Nkkt1bVv07yoSRvnVZfAAC2kxoGs3aPubm5trS0NOsyAACOqaruaq3NrWx3JQYAgM4IcAAAnTlmgKuqU6rqb21FMQAAHNsxA1xr7ckMl7MCAGAb2OgU6oGq+v7xElYAAMzQRgPcjyf5r0mOVNVjVfWFqnpsinUBALCGDX0OXGvta6ZdCAAAG7PhD/Ktqu9N8h3jw8XW2u+stz4AANOxoSnUqnpdkldkuFLCR5K8oqpumGZhAACsbqMjcFckuXA8IzVVdXOGy1ddP63CAABY3fF8kO9pE/e/bpPrAABggzY6AvfzST5UVXckqQzHwl03taoAAFjTMQNcVZ2S5MkkFyf5trH5Z1trn5lmYQAArO6YAa619mRV/Uxr7Z1J9m9BTQAArGOjx8C9t6p+uqrOqapnLN+mWhkAAKva6DFwPzQuXzbR1pI8e3PLAQDgWDZ6DNx1rbV3bEE9AAAcwzGnUMfPfvvnW1ALAAAb4Bg4AIDOOAYOAKAzGwpwrbXzpl0IAAAbs+4UalX9zMT9H1zx3M9PqygAANZ2rGPgrpq4v/LC9Zdtci0AAGzAsQJcrXF/tccAAGyBYwW4tsb91R4DALAFjnUSw7dU1WMZRtu+aryf8fHTploZAACrWjfAtdb2bFUhAABszEY/yBcAgG1CgAMA6IwAtwUOHkxuuGFYAgCcrI1eSosTdPBgcsklyZEjyd69yYEDyfz8rKsCAHpmBG7KFheH8Hb06LBcXJx1RQBA7wS4KVtYGEbe9uwZlgsLs64IAOidKdQpm58fpk0XF4fwZvoUADhZAtwWmJ8X3ACAzWMKFQCgMwIcAEBnBDgAgM4IcAAAnRHgAAA6I8ABAHRGgAMA6IwABwDQGQEOAKAzAhwAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcAAAnRHgAAA6I8ABAHRGgAMA6IwABwDQmakFuKq6qaoeqaoPT7Q9o6pur6r7x+XpY3tV1Rur6lBV3VNVz5t4zTXj+vdX1TUT7d9aVfeOr3ljVdW0+gIAsJ1McwTubUkuW9F2XZIDrbXzkxwYHyfJ5UnOH2/XJnlzMgS+JK9K8vwkFyV51XLoG9f5sYnXrdwXAMCONLUA11r7/SSPrmi+MsnN4/2bk7xwov2WNnhfktOq6llJXpDk9tbao621zyW5Pcll43Nf21p7X2utJbllYlsAADvaVh8Dd2Zr7aHx/meSnDnePyvJpyfWe2BsW6/9gVXaV1VV11bVUlUtHT58+OR6AAAwYzM7iWEcOWtbtK8bW2tzrbW5ffv2bcUuAQCmZqsD3MPj9GfG5SNj+4NJzplY7+yxbb32s1dpBwDY8bY6wO1Psnwm6TVJ3jXRfvV4NurFST4/TrXeluTSqjp9PHnh0iS3jc89VlUXj2efXj2xLQCAHe3UaW24qn49yUKSZ1bVAxnOJn1dkndW1UuTfCrJi8bV353kiiSHkvxZkh9Nktbao1X12iR3juu9prW2fGLET2Y40/WrkvzueAMA2PFqOBRt95ibm2tLS0uzLgMA4Jiq6q7W2tzKdldiAADojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMALfFDh5MbrhhWAIAnIipXQuVr3TwYHLJJcmRI8nevcmBA8n8/KyrAgB6YwRuCy0uDuHt6NFhubg464oAgB4JcFtoYWEYeduzZ1guLMy6IgCgR6ZQt9D8/DBturg4hDfTpwDAiRDgttj8vOAGAJwcU6gAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcAAAnRHgAAA6I8ABAHRGgAMA6IwABwDQGQEOAKAzAhwAQGcEOACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcDNy8GByww3DEgDgeJw66wJ2o4MHk0suSY4cSfbuTQ4cSObnZ10VANALI3AzsLg4hLejR4fl4uKsKwIAeiLAzcDCwjDytmfPsFxYmHVFAEBPTKHOwPz8MG26uDiEN9OnAMDxEOBmZH5ecAMATowpVACAzghwAACdEeAAADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHABAZwQ4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0JmZBLiq+mRV3VtVd1fV0tj2jKq6varuH5enj+1VVW+sqkNVdU9VPW9iO9eM699fVdfMoi8AAFttliNw39lau7C1Njc+vi7Jgdba+UkOjI+T5PIk54+3a5O8ORkCX5JXJXl+kouSvGo59AEA7GTbaQr1yiQ3j/dvTvLCifZb2uB9SU6rqmcleUGS21trj7bWPpfk9iSXbXHNAABbblYBriV5T1XdVVXXjm1nttYeGu9/JsmZ4/2zknx64rUPjG1rtX+Fqrq2qpaqaunw4cOb1QcAgJk4dUb7/fbW2oNV9ZeS3F5VH5t8srXWqqpt1s5aazcmuTFJ5ubmNm27m+XgwWRxMVlYSObnZ10NALDdzSTAtdYeHJePVNVvZziG7eGqelZr7aFxivSRcfUHk5wz8fKzx7YHkyysaF+ccumb7uDB5JJLkiNHkr17kwMHhDgAYH1bPoVaVU+vqq9Zvp/k0iQfTrI/yfKZpNckedd4f3+Sq8ezUS9O8vlxqvW2JJdW1enjyQuXjm1dWVwcwtvRo8NycXHWFQEA290sRuDOTPLbVbW8///SWvtvVXVnkndW1UuTfCrJi8b1353kiiSHkvxZkh9Nktbao1X12iR3juu9prX26NZ1Y3MsLAwjb8sjcAsLs64IANjuqrVtd0jYVM3NzbWlpaVZl/FlHAMHAKymqu6a+Mi1L5rVSQxMmJ8X3ACAjdtOnwMHAMAGCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBnBDgAgM4IcAAAnRHgAAA6I8BtIwcPJjfcMCwBANbiYvbbxMGDySWXJEeOJHv3JgcOuMA9ALA6I3DbxOLiEN6OHh2Wi4uzrggA2K4EuG1iYWEYeduzZ1guLMy6IgBguzKFuk3Mzw/TpouLQ3gzfQoArEWA20bm5wU3AODYTKECAHRGgAMA6IwABwDQGQEOAKAzAhwAQGcEuG3KZbUAgLX4GJFtyGW1AID1GIHbhlxWCwBYjwC3DbmsFgCwHlOo25DLagEA6xHgtimX1QIA1mIKFQCgMwLcNufjRACAlUyhbmM+TgQAWI0RuG3Mx4kAAKsR4LYxHycCAKzGFOo25uNEAIDVCHDb3OTHiRw8KMwBAAJcN5zQAAAscwxcJ5zQAAAsE+A6sfKEhjPO8PlwALBbmULtxOQJDWeckbzylaZTAWC3MgLXkfn55Prrk89+9kvTqY8/nrz61UbiAGA3EeA6tDydesopyZNPJu9973CCgxAHALuDANeh5enU7/7uL4W4I0eSW25xXBwA7AbVWpt1DVtqbm6uLS0tzbqMTTH50SJ79iRVyRNPDPdf8pLk6qsdGwcAPauqu1prc1/RLsD1bfnDff/4j5O3vGU4Li4ZwtzTnpb88i8Px8z58F8A6I8AN9ppAW7Z8mjcn/95svyWnnLKMBr35JNfPiqXuKIDAPRAgBvt1ACXDCHulluSX/3VYSq1aghvTz45PF+VPOUpq0+1JkIdAGw3AtxoJwe4ZcvTqsufFzc5Klc1LCcfbyTUTd4X8ABgawhwo90Q4CatHJVbPtnhyJHjC3XrBbwzznCcHQBMgwA32m0BbtnyqNzyaNrxhLr1At5f/MUwRXvKKclTn/qlkyYmQ10i6AHAiRDgRrs1wK1mI6Fuo6N2yZdOmjh69Euh7tRT1w96awW81dqEPgB2GwFuJMCtb2WoO1bAmwxmyx8qvHzSRLJ+0Fs+O3bldo4V+o4V9FZrE/4A6JEANxLgTs5qAW85KC2fNPH44+uHscmgt1rAO5nRvc0c8VutTRAEYCsJcCMBbromz4BdKwgtB73JK0gcK4RtdHRvs0b8TmQU8GTD4eR2BEUAEgHuiwS47WG9kbyTGd3brBG/4x0FXGtaeaM1rtzOemf6LrdtVmA8nm0LlgBbS4AbCXD92sjo3maN+J3IKODJhMONnum72YHxeLZ9sschrtc2zTC6XcLv5POCMLBRAtxIgNudjnfE73hHATcrUB3rTN9pBcaNbPtkjkM8kcC4GdveLuH3REZYewij9qcv9jf9f8gEuJEAx8lYbxRwM36ZHOtM31mGkJM5DnGWYXSr97eR11StP8LaQxi1P32xv2Fm4sCB6YY4AW4kwLHdbXS0cKv/Iz2Z4xB30i/sk9n28YywbrRt1mHU/vRlN+9vz57kta9Nrr8+UyPAjQQ4OHEnehziLMPodgi/y20bHWHtIYzan77YnxG4LSXAAbO0Gcdjbocwan/6Yn+OgdtSAhwA0Iu1AtwpsygGAIATJ8ABAHRGgAMA6IwABwDQGQEOAKAz3Qe4qrqsqj5eVYeq6rpZ1wMAMG1dB7iq2pPkTUkuT3JBkhdX1QWzrQoAYLq6DnBJLkpyqLX2idbakSS3JrlyxjUBAExV7wHurCSfnnj8wNj2Zarq2qpaqqqlw4cPb1lxAADT0HuA25DW2o2ttbnW2ty+fftmXQ4AwEnpPcA9mOScicdnj20AADtW7wHuziTnV9V5VbU3yVVJ9s+4JgCAqTp11gWcjNbaE1X18iS3JdmT5KbW2n0zLgsAYKq6DnBJ0lp7d5J3z7oOAICt0vsUKgDAriPAAQB0plprs65hS1XV4SSfmvJunpnkT6a8j+1M/3dv/3dz3xP938393819T/R/mv3/y621r/gMtF0X4LZCVS211uZmXces6P/u7f9u7nui/7u5/7u574n+z6L/plABADojwAEAdEaAm44bZ13AjOn/7rWb+57o/27u/27ue6L/W95/x8ABAHTGCBwAQGcEuE1WVZdV1cer6lBVXTfreqapqs6pqjuq6iNVdV9VvWJsf3VVPVhVd4+3K2Zd67RU1Ser6t6xn0tj2zOq6vaqun9cnj7rOqehqv7KxHt8d1U9VlWv3Mnvf1XdVFWPVNWHJ9pWfb9r8Mbxd8E9VfW82VV+8tbo+y9W1cfG/v12VZ02tp9bVf9v4nvgP8ys8E2yRv/X/F6vquvH9/7jVfWC2VS9edbo/zsm+v7Jqrp7bN9R7/86f+tm+rNvCnUTVdWeJH+Y5O8leSDJnUle3Fr7yEwLm5KqelaSZ7XWPlhVX5PkriQvTPKiJH/aWvs3s6xvK1TVJ5PMtdb+ZKLtF5I82lp73RjiT2+t/eysatwK4/f+g0men+RHs0Pf/6r6jiR/muSW1to3j22rvt/jH/OfSnJFhq/Lv2utPX9WtZ+sNfp+aZLfG69L/fokGft+bpLfWV5vJ1ij/6/OKt/rVXVBkl9PclGSr0/y3iTf1Fo7uqVFb6LV+r/i+Tck+Xxr7TU77f1f52/dj2SGP/tG4DbXRUkOtdY+0Vo7kuTWJFfOuKapaa091Fr74Hj/C0k+muSs2Va1LVyZ5Obx/s0ZftB3ukuS/K/W2rQ/JHumWmu/n+TRFc1rvd9XZvhj11pr70ty2viHoEur9b219p7W2hPjw/clOXvLC9sia7z3a7kyya2ttcdba3+U5FCGvw/dWq//VVUZ/nH/9S0taous87dupj/7AtzmOivJpyceP5BdEmjG/7iem+T9Y9PLx6Hjm3bqFOKoJXlPVd1VVdeObWe21h4a738myZmzKW1LXZUv/+W9W97/ZO33e7f9PnhJkt+deHxeVX2oqv57Vf2dWRW1BVb7Xt9t7/3fSfJwa+3+ibYd+f6v+Fs30599AY6TVlVfneQ3k7yytfZYkjcneU6SC5M8lOQNs6tu6r69tfa8JJcnedk4zfBFbThGYUcfp1BVe5N8b5L/Ojbtpvf/y+yG93s1VfVzSZ5I8vax6aEk39Bae26Sf5rkv1TV186qvinatd/rK7w4X/4P3I58/1f5W/dFs/jZF+A214NJzpl4fPbYtmNV1VMyfEO/vbX2W0nSWnu4tXa0tfZkkrek86mD9bTWHhyXjyT57Qx9fXh5uHxcPjK7CrfE5Uk+2Fp7ONld7/9orfd7V/w+qKofSfI9SX54/COWcerws+P9u5L8ryTfNLMip2Sd7/Vd8d4nSVWdmuQfJHnHcttOfP9X+1uXGf/sC3Cb684k51fVeeOoxFVJ9s+4pqkZj3t4a5KPttZ+aaJ9cq7/+5J8eOVrd4Kqevp4QGuq6ulJLs3Q1/1JrhlXuybJu2ZT4Zb5sv++d8v7P2Gt93t/kqvHM9IuznCA90OrbaBXVXVZkp9J8r2ttT+baN83ntiSqnp2kvOTfGI2VU7POt/r+5NcVVVPrarzMvT/A1td3xb57iQfa609sNyw097/tf7WZdY/+601t028ZTjr5A8z/Mfxc7OuZ8p9/fYMQ8b3JLl7vF2R5D8nuXds35/h7J2Z1zuF/j87yR+Mt/uW3+8kZyQ5kOT+DGefPWPWtU7xa/D0JJ9N8nUTbTv2/c8QVB9K8hcZjmt56Vrvd5JK8qbxd8G9Gc5WnnkfNrnvhzIc67P88/8fxnW/f/yZuDvJB5P8/VnXP6X+r/m9nuTnxvf+40kun3X90+j/2P62JD+xYt0d9f6v87dupj/7PkYEAKAzplABADojwAEAdEaAAwDojAAHANAZAQ4AoDMCHDAzVdXGi2AvP/7p8QLhm7Htt1XVD2zGto6xnx+sqo9W1R0r2r++qn5jvH/heIHrzdrnaVX1k6vtC9gdBDhglh5P8g+q6pmzLmTS+OnyG/XSJD/WWvvOycbW2v9urS0HyAszfG7UZtVwWpIvBrgV+wJ2AQEOmKUnktyY5J+sfGLlCFpV/em4XBgvkP2uqvpEVb2uqn64qj5QVfdW1XMmNvPdVbVUVX9YVd8zvn5PVf1iVd05XoT8xye2+z+qan+Sj6xSz4vH7X+4ql4/tv2rDB/y+daq+sUV6587rrs3yWuS/FBV3V1VPzRexeOmseYPVdWV42t+pKr2V9XvJTlQVV9dVQeq6oPjvq8cN/+6JM8Zt/eLy/sat/G0qvrVcf0PVdV3Tmz7t6rqv1XV/VX1CxNfj7eNtd5bVV/xXgDbz/H8lwkwDW9Kcs9yoNigb0ny15I8muESPf+ptXZRVb0iyU8leeW43rkZrk/5nCR3VNU3Jrk6w6Vtvq2qnprkf1bVe8b1n5fkm1trfzS5s6r6+iSvT/KtST6X5D1V9cLW2muq6ruS/HRrbWm1QltrR8agN9dae/m4vZ9P8nuttZdU1WlJPlBV752o4W+21h4dR+G+r7X22DhK+b4xYF431nnhuL1zJ3b5smG37W9U1V8da12+DuWFSZ6bYeTz41X175P8pSRntda+edzWaet83YFtwggcMFOttceS3JLkHx/Hy+5srT3UWns8w+VqlgPYvRlC27J3ttaebK3dnyHo/dUM16y9uqruTvL+DJfDOX9c/wMrw9vo25IsttYOt9aeSPL2JN9xHPWudGmS68YaFpM8Lck3jM/d3lp7dLxfSX6+qu7JcKmes5KceYxtf3uSX0uS1trHknwqX7qQ+IHW2udba3+eYZTxL2f4ujy7qv59Ddc2fewk+gVsESNwwHbwyxmumfirE21PZPwns6pOSbJ34rnHJ+4/OfH4yXz577WV1wpsGULRT7XWbpt8oqoWkvzfEyn+BFSS72+tfXxFDc9fUcMPJ9mX5Ftba39RVZ/MEPZO1OTX7WiSU1trn6uqb0nygiQ/keRFSV5yEvsAtoAROGDmxhGnd2Y4IWDZJzNMWSbJ9yZ5ygls+ger6pTxuLhnZ7iw+G1J/lFVPSVJquqbqurpx9jOB5L83ap6ZlXtSfLiJP/9OOr4QpKvmXh8W5Kfqqoaa3juGq/7uiSPjOHtOzOMmK22vUn/I0Pwyzh1+g0Z+r2qcWr2lNbabyb5lxmmcIFtToADtos3JJk8G/UtGULTHySZz4mNjv1xhvD1u0l+Ypw6/E8Zpg8/OB74/x9zjNmI1tpDGY47uyPJHyS5q7X2ruOo444kFyyfxJDktRkC6T1Vdd/4eDVvTzJXVfdmOHbvY2M9n81w7N6HV548keRXkpwyvuYdSX5knGpey1lJFsfp3F9Lcv1x9AuYkWpt5QwDAADbmRE4AIDOCHAAAJ0R4AAAOiPAAQB0RoADAOiMAAcA0BkBDgCgMwIcAEBn/j/ZBI4Kg8G9FAAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Things to try out:\n",
        "\n",
        "1. Change batch size in mini-batch gradient descent.\n",
        "2. Test all the three out on real datasets.\n",
        "3. Compare the effects of changing learning rate by the same amount in Batch GD, SGD and Mini-batch GD."
      ],
      "metadata": {
        "id": "0neTARjKUoP4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#**Answer For 1st Question**"
      ],
      "metadata": {
        "id": "mjxaCQqvuUCd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lr = 0.1\n",
        "n_iter = 200\n",
        "\n",
        "cost_history_list = []\n",
        "\n",
        "# batch size\n",
        "batch_size_list = [1, 20, 40, 60, 80, 100]\n",
        "\n",
        "for bt_size in batch_size_list:\n",
        "  \"\"\"\n",
        "  When,\n",
        "    bt_sze = 1 -> Stocastic gradient\n",
        "    bt_sze = m (100) -> Batch gradient\n",
        "  \"\"\"\n",
        "  y_pred = np.random.randn(2,1)\n",
        "  y_pred, cost_history = minibatch_gradient_descent(X, y, y_pred, learning_rate=lr, iterations=n_iter, batch_size=bt_size)\n",
        "\n",
        "  cost_history_list.append(cost_history)\n",
        "\n",
        "  print(f\"Batch Size = {bt_size}\")\n",
        "  print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "  print('Final error: {:0.3f}'.format(cost_history[-1]))\n",
        "  print(\" \")"
      ],
      "metadata": {
        "id": "Bg39V9qbubDs",
        "outputId": "bba93207-b475-46e8-b218-5aeaba82748a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Batch Size = 1\n",
            "y_pred[0]: 4.248\n",
            "y_pred[1]: 2.741\n",
            "Final error: 40.253\n",
            " \n",
            "Batch Size = 20\n",
            "y_pred[0]: 4.188\n",
            "y_pred[1]: 2.797\n",
            "Final error: 806.147\n",
            " \n",
            "Batch Size = 40\n",
            "y_pred[0]: 4.154\n",
            "y_pred[1]: 2.824\n",
            "Final error: 1448.594\n",
            " \n",
            "Batch Size = 60\n",
            "y_pred[0]: 4.191\n",
            "y_pred[1]: 2.791\n",
            "Final error: 2003.070\n",
            " \n",
            "Batch Size = 80\n",
            "y_pred[0]: 4.231\n",
            "y_pred[1]: 2.756\n",
            "Final error: 2749.832\n",
            " \n",
            "Batch Size = 100\n",
            "y_pred[0]: 4.184\n",
            "y_pred[1]: 2.797\n",
            "Final error: 4034.362\n",
            " \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "fig, ax = plt.subplots(figsize=(10,8))\n",
        "\n",
        "ax.set_ylabel('Error')\n",
        "ax.set_xlabel('Number of iterations')\n",
        "y_pred = np.random.randn(2,1)\n",
        "\n",
        "for bt_size, cost_history in zip(batch_size_list, cost_history_list):\n",
        "  ax.plot(range(n_iter), cost_history, label=str(bt_size))\n",
        "\n",
        "plt.ylim([0, 10000])\n",
        "\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "OEGJYNrAuoBw",
        "outputId": "a77e48d2-3719-459a-8380-f4ed6d71b375",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 501
        }
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 720x576 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnAAAAHkCAYAAAC+DBJrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAADGq0lEQVR4nOzdd3xb9b3/8dfRlvcesbOXkzhkkrAhhA2FDkYLtw0tLb3dpXvdFm5bCp2UXtr+2lJIC4VSSoEyyggjjATIHnYcO3HivbdszfP9/XGkY9mWHSexYyX+PB8PP2QfHR19ZcvSW5/vOJpSCiGEEEIIcfKwTHQDhBBCCCHE0ZEAJ4QQQghxkpEAJ4QQQghxkpEAJ4QQQghxkpEAJ4QQQghxkpEAJ4QQQghxkhm3AKdp2p81TWvSNG1P1LYMTdNe0jStPHyZHt6uaZp2r6ZpFZqm7dI0bXnUbdaF9y/XNG1d1PYVmqbtDt/mXk3TtPF6LEIIIYQQ8WQ8K3APApcN2vYtYINSai6wIfwzwOXA3PDXrcDvwAh8wA+A1cAq4AeR0Bfe51NRtxt8X0IIIYQQp6RxC3BKqY1A26DN1wDrw9+vB94ftf0vyrAZSNM0LR+4FHhJKdWmlGoHXgIuC1+XopTarIyViP8SdSwhhBBCiFPaiR4Dl6uUqg9/3wDkhr8vAKqj9qsJbxtpe02M7UIIIYQQpzzbRN2xUkppmnZCzuOladqtGF2zJCYmrigqKjoRdzs6bQchFIDs+bRU1qJbksmZnjJgl+bDlTgTEknJzpmgRgohhBBiImzdurVFKZU9ePuJDnCNmqblK6Xqw92gTeHttcDUqP0Kw9tqgQsGbX8tvL0wxv4xKaX+APwBYOXKlWrLli3H9yjG0iMfgc5q+O83eeCmb9CbfBmf+e0aLJb+ORl//PwtFBYt5PLPf3UCGyqEEEKIE03TtMOxtp/oLtSngchM0nXAU1HbPxaejXoG0Bnuan0BuETTtPTw5IVLgBfC13VpmnZGePbpx6KOdXLRLKCHwt8bl3pQH7CLzW4nGAye6JYJIYQQIk6NWwVO07RHMKpnWZqm1WDMJr0LeEzTtFuAw8D14d2fA64AKoBe4OMASqk2TdN+CLwX3u9/lVKRiRGfxZjp6gaeD3+dfCy2qABnBLdQSA34w1jtdkIB/4lvmxBCCCHi0rgFOKXUR4a5am2MfRXwuWGO82fgzzG2bwGKj6eNccFiBT1cXRu2AucgFAic6JYJIYQQIk5N2CQGEWaxgRpUgQsOnNthtdsJSgVOCCGEIBAIUFNTg9frneimjCmXy0VhYSF2u31U+0uAm2jRXagWI7jpoYEVOKvdjr+v90S3TAghhIg7NTU1JCcnM2PGDE6VkzAppWhtbaWmpoaZM2eO6jZyLtSJplmiulAjFbihAS4UkEkMQgghhNfrJTMz85QJbwCappGZmXlUVUUJcBNtQAUudheqzSZdqEIIIUTEqRTeIo72MUmAm2gDJjEM04XqcBAKyiQGIYQQIh584hOfICcnh+LiiZtLKQFuokVPYrAOM4nBZifklwqcEEIIEQ9uvvlm/vOf/0xoGyTATbRYkxgGLyPikIV8hRBCiHhx3nnnkZGRMaFtkFmoEy1qEoNmUaBDaHAXqs0u68AJIYQQg9zx772U1HWN6TEXTknhB+9bNKbHHA9SgZto0RU4q3ERexaqdKEKIYQQwiAVuIkWPYnBpoEPQr6B3aU2uwM9FELXQ1gs1glopBBCCBF/ToZK2XiRCtxEs9gABboONmMMXMg/MMBZw6sySzeqEEIIIUAC3MSLVNRUCM1mrAEzOMDZzAAnExmEEEKIifaRj3yEM888k7KyMgoLC7n//vtPeBukC3WiaeEApweNLlRGqMDJWnBCCCHEhHvkkUcmuglSgZtwlnCG1kNgDwe4QWPgrDYjwAVlLTghhBBCIAFu4ln6K3AWm/HnGNxVanU4jO1SgRNCCCEEEuAmXqQCp3RwhAOcPzRgF5tU4IQQQggRRQLcRIuqwGn24SpwMgZOCCGEEP0kwE20qEkMFocNTYUIBYaeiQFkGREhhBBCGCTATbSoSQwWuwNNDw2pwNnsxhi4oAQ4IYQQQiABbuJFT2JwOLCoIMHAwDFwspCvEEIIER+qq6tZs2YNCxcuZNGiRfz6178GoK2tjYsvvpi5c+dy8cUX097ePq7tkAA30aImMVhtRgUuOHgMnBngZBKDEEIIMZFsNhu/+MUvKCkpYfPmzdx3332UlJRw1113sXbtWsrLy1m7di133XXXuLZDAtxEG1CBc8aswNmkAieEEELEhfz8fJYvXw5AcnIyCxYsoLa2lqeeeop169YBsG7dOp588slxbYeciWGiRU1isDmcRgUuGLsLVcbACSGEEFGe/xY07B7bY+YthstHVz07dOgQ27dvZ/Xq1TQ2NpKfn28cIi+PxsbGsW3XIFKBm2hRkxisDicWPUgoOHAWamQSg1TghBBCiPjQ09PDhz70Ie655x5SUlIGXKdpGpqmjev9SwVuokV3odqNSQyDA5ycC1UIIYSIYZSVsrEWCAT40Ic+xE033cQHP/hBAHJzc6mvryc/P5/6+npycnLGtQ1SgZtoUZMY7E4Xmh5CD8VeB07OxCCEEEJMLKUUt9xyCwsWLOArX/mKuf3qq69m/fr1AKxfv55rrrlmXNshFbiJFlWBs9mdWFQfoZAasItU4IQQQoj48NZbb/HXv/6VxYsXs3TpUgDuvPNOvvWtb3H99ddz//33M336dB577LFxbYcEuIkWNYnB6jAqcKHQoF00DavNJpMYhBBCiAl2zjnnoJSKed2GDRtOWDukC3WiRU1isDtdWFQQXR/6xLDaHTKJQQghhBCABLiJF9WFane4sehBVGjobla7XQKcEEIIIQAJcBMvehKDw42mQuj60KnHNruDoJyJQQghhBBIgJt40ZMYnC4sehBdDQ1wVrtNKnBCCCGEACTATbyoSQwOuwtUCBUrwNmkC1UIIYQQBglwEy1qEoPD4gAVjN2F6nDIMiJCCCGEACTATbzoSQxWOxBCxfizWG12WUZECCGEiBOhUIhly5Zx1VVXAVBZWcnq1auZM2cON9xwA/5xXnxfAtxEi5rEYLPYQAVRauifxeawE5JJDEIIIURc+PWvf82CBQvMn7/5zW9y2223UVFRQXp6Ovfff/+43r8EuIkWXYGz2EELDluBkzFwQgghxMSrqanh2Wef5ZOf/CRgnF7rlVde4dprrwVg3bp1PPnkk+PaBjkTw0SLnsRgdaAIobCilELT+sfCWe3ShSqEEEJEu/vdu9nXtm9Mj1mUUcQ3V31zxH2+/OUv89Of/pTu7m4AWltbSUtLw2YzYlVhYSG1tbVj2q7BpAI30aLPxGCxowgaP+qDz4cqZ2IQQgghJtozzzxDTk4OK1asmNB2SAVuokV1oVo1K2CchiEU0LFa+/O1zW6XWahCCCFElCNVysbDW2+9xdNPP81zzz2H1+ulq6uLL33pS3R0dBAMBrHZbNTU1FBQUDCu7ZAK3ESLqsBpmobSQuEfB1fg7ATHeUaLEEIIIUb2k5/8hJqaGg4dOsSjjz7KhRdeyMMPP8yaNWt4/PHHAVi/fj3XXHPNuLZDAtxEMwNcMLwhXIEL6gN2s0oFTgghhIhbd999N7/85S+ZM2cOra2t3HLLLeN6f9KFOtGsduMyZFTXhqvA2ewOQn4JcEIIIUS8uOCCC7jgggsAmDVrFu++++4Ju2+pwE00q8O4DFfglGZU3mJV4ILBAEoNDHZCCCGEmHwkwE00ixXQzAocFiO46cFBY+BsdlAKPRRECCGEEJObBLh4YHVAyOgeVZbYFTib3ehqlaVEhBBCCCEBLh5Y7WaAi1TgQqGhXaiALOYrhBBCCAlwccFiAz1SgTO6Tod0odqNsXJSgRNCCCGEBLh4YHWYY+C0cIAbXIGTLlQhhBBCREiAiwdWO0QmJ9iGq8BFulBlMV8hhBBiInV0dHDttddSVFTEggUL2LRpE21tbVx88cXMnTuXiy++mPb29nFtgwS4eGC1968DF/6LDF1GRLpQhRBCiHjwpS99icsuu4x9+/axc+dOFixYwF133cXatWspLy9n7dq13HXXXePaBglw8cBiN8fAaeGllYfMQrXZwtslwAkhhBATpbOzk40bN5pnWnA4HKSlpfHUU0+xbt06ANatW8eTTz45ru2QMzHEg6hlRLACKsa5UB1GBS4oZ2MQQgghAGi48058pfvG9JjOBUXkfec7w15fWVlJdnY2H//4x9m5cycrVqzg17/+NY2NjeTn5wOQl5dHY2PjmLZrMKnAxQOrzQxww1XgrLbwJAapwAkhhBATJhgMsm3bNj7zmc+wfft2EhMTh3SXapqGpmnj2g6pwMUDqyOqC1WDQIxzoUYqcDKJQQghhAAYsVI2XgoLCyksLGT16tUAXHvttdx1113k5uZSX19Pfn4+9fX15OTkjGs7pAIXDyz2qAqc8ScJBUIDdjErcDKJQQghhJgweXl5TJ06lbKyMgA2bNjAwoULufrqq1m/fj0A69ev55prrhnXdkgFLh5Y7RD0AaA5jJJryB8ctIsEOCGEECIe/OY3v+Gmm27C7/cza9YsHnjgAXRd5/rrr+f+++9n+vTpPPbYY+PaBglw8cBqB183ABabFRga4GQhXyGEECI+LF26lC1btgzZvmHDhhPWBulCjQfRY+DssQOcnAtVCCGEEBES4OKBpX8WqtXhQNODhPwDx8DZzIV8ZRKDEEIIMdlJgIsHUevAWex2LHqQwKD13mQMnBBCCCEiJMDFA6s9KsA50FRwSFepxWpF0yyyDpwQQgghJMDFBWv/qbQsDjsWPURg0Bg4AKvDLmPghBBCCCEBLi5Y+k9mb7E70VSIYGBogLPZ7NKFKoQQQggJcHHBaodQMPytA4seHLKQLxgTHORMDEIIIcTE+tWvfsWiRYsoLi7mIx/5CF6vl8rKSlavXs2cOXO44YYb8PvH9/1aAlw8sEZV4BwOLCpEMDg0wNnsdkLj/IQQQgghxPBqa2u599572bJlC3v27CEUCvHoo4/yzW9+k9tuu42KigrS09O5//77x7UdEuDigaV/DJzN7kTTA0NOZg9gd7oI+HwnunVCCCGEiBIMBunr6yMYDNLb20t+fj6vvPIK1157LQDr1q3jySefHNc2yJkY4oHVAXoQlMLqcGFRIfRYAc7lwu/tm4AGCiGEEPHnjcf201LdM6bHzJqaxLnXzxv2+oKCAr72ta8xbdo03G43l1xyCStWrCAtLQ2bzYhVhYWF1NbWjmm7BpMKXDywhnN0KIDN4UDTQ4RCashuUoETQgghJlZ7eztPPfUUlZWV1NXV4fF4+M9//nPC2yEVuHhgNc6ygB4IV+A8BGMFOJebvu6uE9w4IYQQIj6NVCkbLy+//DIzZ84kOzsbgA9+8IO89dZbdHR0EAwGsdls1NTUUFBQMK7tkApcPLAYZ1kg5MfucGHRQ+gxApzD5SLg9Z7gxgkhhBAiYtq0aWzevJne3l6UUmzYsIGFCxeyZs0aHn/8cQDWr1/PNddcM67tkAAXD6yRABfE5nShqSD60Emo4S5UCXBCCCHERFm9ejXXXnsty5cvZ/Hixei6zq233srdd9/NL3/5S+bMmUNrayu33HLLuLZDulDjgbW/Amezu7DoQfShcxjCkxgkwAkhhBAT6Y477uCOO+4YsG3WrFm8++67J6wNUoGLB1Fj4OxON5oeQunakN3sLqMCp9TQ7lUhhBBCTB4S4OKBOQYugM3hwqKC6LECnNMFShH0y0xUIYQQYjKTABcPrP0Bzu50Y9FDKDU0wDlcbgCZyCCEEEJMchLg4oF14CxUTQWH7UIFZBycEEIIMclJgIsH5hi4IE5nglGBi/GniQQ4mYkqhBBCTG4TEuA0TbtN07S9mqbt0TTtEU3TXJqmzdQ07R1N0yo0Tfu7pmmO8L7O8M8V4etnRB3n2+HtZZqmXToRj2VMWCJnYvBjd7jRVAClhv5pHM5wgJPTaQkhhBCT2gkPcJqmFQBfBFYqpYoBK/Bh4G7gV0qpOUA7EFlA5RagPbz9V+H90DRtYfh2i4DLgN9qmmY9kY9lzESNgXPYnKBCgAVdHzjb1CZdqEIIIcSE+8QnPkFOTg7FxcXmtra2Ni6++GLmzp3LxRdfTHt7OwBKKb74xS8yZ84cTjvtNLZt2zYmbZioLlQb4NY0zQYkAPXAhcDj4evXA+8Pf39N+GfC16/VNE0Lb39UKeVTSlUCFcCqE9P8MRbpQg0FsFqsQBAAPTRwMThzEoN0oQohhBAT5uabbx5y/tO77rqLtWvXUl5eztq1a7nrrrsAeP755ykvL6e8vJw//OEPfOYznxmTNpzwAKeUqgV+DlRhBLdOYCvQoZQKhnerASInESsAqsO3DYb3z4zeHuM2J5dIF6oeCG8wTsOgBwdW4MwxcFKBE0IIISbMeeedR0ZGxoBtTz31FOvWrQNg3bp1PPnkk+b2j33sY2iaxhlnnEFHRwf19fXH3YYTfiYGTdPSMapnM4EO4B8YXaDjeZ+3AreCcQ6zuGNW4PwAqHCACwUHVuDsMgZOCCGEML364B9oOnxwTI+ZM30Wa26+9ahv19jYSH5+PgB5eXk0NjYCUFtby9SpU839CgsLqa2tNfc9VhPRhXoRUKmUalZKBYAngLOBtHCXKkAhUBv+vhaYChC+PhVojd4e4zYDKKX+oJRaqZRamZ2dPdaP5/hFjYEDQIsEuIEVOIeMgRNCCCHinqZpGKO9xs9EnAu1CjhD07QEoA9YC2wBXgWuBR4F1gFPhfd/OvzzpvD1ryillKZpTwN/0zTtl8AUYC5w4k5CNpYiAU43epAjFbjBY+BsTicgXahCCCEEcEyVsvGSm5tLfX09+fn51NfXk5OTA0BBQQHV1f0jvmpqaigoOP4RXxMxBu4djMkI24Dd4Tb8Afgm8BVN0yowxrjdH77J/UBmePtXgG+Fj7MXeAwoAf4DfE4pFTqBD2XsWPoX8gWiKnADA5zFYsXmcMokBiGEECLOXH311axfb8y5XL9+Pddcc425/S9/+QtKKTZv3kxqaupxd5/CxFTgUEr9APjBoM0HiTGLVCnlBa4b5jg/Bn485g080aJmoQKgGcFNDw09ab3d5ZIxcEIIIcQE+shHPsJrr71GS0sLhYWF3HHHHXzrW9/i+uuv5/7772f69Ok89thjAFxxxRU899xzzJkzh4SEBB544IExacOEBDgxyKAxcGqYChwY4+CkC1UIIYSYOI888kjM7Rs2bBiyTdM07rvvvjFvg5xKKx6YY+DCFTiLUXkbPIkBjJmoMolBCCGEmNwkwMWDwWPgLOEu1BgVOLvLJWPghBBCiElOAlw8MLtQw7NQIxW4UKwA55YuVCGEEGKSkwAXDyxW0CwxKnCxu1BlEoMQQojJTKmh748nu6N9TBLg4oXV0T8GzmpcxKrAOVwu/NKFKoQQYpJyuVy0traeUiFOKUVrayuu8IL9oyGzUOOFxW7OQtXCXagxK3AyC1UIIcQkVlhYSE1NDc3NzRPdlDHlcrkoLCwc9f4S4OKF1da/Dlz4rxJrGRGjC1UCnBBCiMnJbrczc+bMiW7GhJMu1HhhdZhj4LTwXyVmgHO5Cfi8KH3odUIIIYSYHCTAxQuL3TwXqmY3ToAb60wMkRPaB/3+E9c2IYQQQsQVCXDxwmrvr8AdoQsVwC8zUYUQQohJSwJcvLBGTWKwGX+W4c6FCsg4OCGEEGISkwAXL6wOswvVEg5wsc+F6gakAieEEEJMZhLg4oXFZnahWhw2UDrBQGjIbnanE4CAz3dCmyeEEEKI+CEBLl5YHf1dqHY7Fj1IIBAcsps9XIGTszEIIYQQk5cEuHgRPQYuEuB8gSG7yRg4IYQQQkiAixdWu3kqLYvNjkUPEPCPEODkdFpCCCHEpCUBLl5YopYRcYQDXIwKXP8kBglwQgghxGQlAS5eWB0QCs9CtTuw6EGC/lhj4CJdqDIGTgghhJisJMDFC2vULFS7A4sKEvTHmIXqiMxClQqcEEIIMVlJgIsXVoc5Bs7qcGDRAzGXEdEsFmxOp3ShCiGEEJOYBLh4YemfhWq1O7HoAUKB2Cesd7jc0oUqhBBCTGIS4OKF1RYV4IwxcHqMMzGAsZivLOQrhBBCTF4S4OKF1WGOgbM6XVj0IKHg0HOhgrGYr1TghBBCiMlLAly8sNjNc6Hawl2o+tBJqIAxE1XGwAkhhBCTlwS4eGHtXwfO6giPgRuuAud0ySxUIYQQYhKTABcvrIMnMQRRuhZzV4fLJafSEkIIISYxCXDxwuoAFQJdxx4eA6cPXUUEkDFwQgghxGQnAS5eWGzGpR7A5nBh0QMoPfafR2ahCiGEEJObBLh4YXUYl6GBAU6poePg7C43fqnACSGEEJOWBLh4YbUblyE/dofRhQqgh4YGOIfLRdDnQx+uj1UIIYQQpzQJcPEiEuD0IHZnghngYp2Nwe40TmgflG5UIYQQYlKSABcvLFEVOKfRhQoQjBXgXG4AGQcnhBBCTFIS4OJF1Bg4hzPBDHChGKfTcriMCpyMgxNCCCEmJwlw8cIcAxfAEj4XqvFjrApcOMD19p6w5gkhhBAifkiAixfmGLgAmt2ORYUDXIwKnCspGQCvp+eENU8IIYQQ8UMCXLyIGgOn2WwjjoEzA1yPBDghhBBiMpIAFy/MLtSgUYGLjIGLEeDcZoDrOmHNE0IIIUT8kAAXL6LWgcNq7R8DF6ML1ZmUBEgFTgghhJisJMDFC0vUGDhNQzHCJAaHE5vDSV9P94lsoRBCCCHihAS4eBG1jAiAbjHOshBrDByAKzkZrwQ4IYQQYlKSABcvrOGT2YcDHDYjuMXqQgVwJyZJgBNCCCEmKQlw8cKswPkBUEcIcK4kqcAJIYQQk5UEuHhh6T8XKoBuFuRGCnAyiUEIIYSYjCTAxYuoMzEAaDYNGHkMXF+3LCMihBBCTEYS4OJF9DIigDLnNIxcgVNKnYjWCSGEECKOSICLF5ExcOEFfDWnDZROKBiKubsrMQk9FCTg856oFgohhBAiTkiAixeWgbNQNZcTTQXx+2MHOHdyCgDebpnIIIQQQkw2EuDixaB14DSnA00P4vP7Y+7uCp+NQRbzFUIIISYfCXDxYtAYOM3pwqIH8PsDMXfvP6G9BDghhBBispEAFy8GLSNicbux6AECgWDM3ftPaC9LiQghhBCTjQS4eGGxgGY1K3BWlxurHiTgjx3g+itwspSIEEIIMdlIgIsnVrs5Bs6aMHKAc4bHwEkFTgghhJh8JMDFE6vDDHA2VwIWPUDQFzvA2R1ObA6nTGIQQgghJiEJcPHEYjPXgbO5E7HoAULDVODAOBuDLCMihBBCTD4S4OKJ1WGOgbMnJGHRg4QCsdeBA3AnJuH1SIATQgghJhsJcPHEaoeQUXFzJCSGA9zwp8oyTqclAU4IIYSYbCTAxROr3azAORKSsegBVHCEAJecLJMYhBBCiElIAlw8sdjNMXDOcIDThx8Chyspmb5uWUZECCGEmGwkwMUTqyOqCzUFix5EhbRhdze6UHtQavgqnRBCCCFOPRLg4onVZnah2twJRoDThw9w7qRk9FCQgM97oloohBBCiDggAS6eWB1mF6rF5cSiB0Af/k9kno1BlhIRQgghJhUJcPHE0n8mBi18LlSUddguUlf4bAyymK8QQggxuUiAiydRp9KyOJ1Y9CCgoYeGC3CR86FKgBNCCCEmEwlw8SRqGRGjAmdMaAgF9Ji7uyXACSGEEJOSBLh4YnUQWTfEqMAZ1bjgMAFOKnBCCCHE5CQBLp5Y+mehanY7GkaACwWPFOBkMV8hhBBiMpEAF0+ixsABKIsR3IbrQrU5HNicTpnEIIQQQkwyEuDiidUxIMChhQPcMBU4CC/mK8uICCGEEJOKBLh4YrGZ68ABKJsR3IYbAwfgTkyir0dOpyWEEEJMJhLg4onVYY6BA9CsxvIhI1Xg3Klp9HZ2jHfLhBBCCBFHJMDFE6vdPBeq8bNxMdwYOIDkzCx6WlvGuWFCCCGEiCcS4OJJ1DpwAJrduAwEgsPcIBzgOtoJBYffRwghhBCnFglw8STShRo+dZbFbpzI3uv1DXuT5MwsUApPR9sJaaIQQgghJp4EuHhicwKq/3yoDqMP1esfIcBlZAHQ3SLdqEIIIcRkIQEuntjcxmXQC4DVaQPA6/MPdwujAgd0t0mAE0IIISYLCXDxxOY0LoNGxc3qMAKczz98gEsKBziZyCCEEEJMHhLg4onNZVwG+4wf3cYshpECnDMhEbvLTbcEOCGEEGLSmJAAp2lamqZpj2uatk/TtFJN087UNC1D07SXNE0rD1+mh/fVNE27V9O0Ck3TdmmatjzqOOvC+5drmrZuIh7LmDIDnFGBs7sdAPj9geFugaZpJGdkSheqEEIIMYlMVAXu18B/lFJFwBKgFPgWsEEpNRfYEP4Z4HJgbvjrVuB3AJqmZQA/AFYDq4AfRELfScseCXDGGDib2xgT5/cNH+AAkrOypQInhBBCTCInPMBpmpYKnAfcD6CU8iulOoBrgPXh3dYD7w9/fw3wF2XYDKRpmpYPXAq8pJRqU0q1Ay8Bl52wBzIeIhW4gBHgHO5ELCE/Ae/wXagASRmZMgZOCCGEmEQmogI3E2gGHtA0bbumaX/SNC0RyFVK1Yf3aQByw98XANVRt68Jbxtu+8nLnMQQrsAlJGLRgwSPEOCSM7NlMV8hhBBiEpmIAGcDlgO/U0otAzz0d5cCoJRSgBqrO9Q07VZN07Zomralubl5rA479sxlRCJj4JKw6AGCvpGDWXJmpizmK4QQQkwiExHgaoAapdQ74Z8fxwh0jeGuUcKXTeHra4GpUbcvDG8bbvsQSqk/KKVWKqVWZmdnj9kDGXNmBc6YhepISMSiguj+0Ig3S840HpMs5iuEEEJMDic8wCmlGoBqTdPmhzetBUqAp4HITNJ1wFPh758GPhaejXoG0Bnuan0BuETTtPTw5IVLwttOXoNmoToTU7DoAUL+4U9mD5CckQnIYr5CCCHEZGGboPv9AvCwpmkO4CDwcYww+ZimabcAh4Hrw/s+B1wBVAC94X1RSrVpmvZD4L3wfv+rlDq5+xAHzUK1uxOx6C3oAeuIN0vOClfgZCKDEEIIMSlMSIBTSu0AVsa4am2MfRXwuWGO82fgz2PauIlkzkI1ulAtbjcWPYAKOEa8mcOdgN3llpmoQgghxCQhZ2KIJ4NOpWVxubDoQdQRJpdqmkZyZpZ0oQohhBCThAS4eDLoZPaa02VU4ELaEW+anJklXahCCCHEJCEBLp5Y7YDWX4FzGxU4RhngpAtVCCGEmBwkwMUTTTPGwYWXEYlU4NCP/GdKysiSxXyFEEKISUICXLyxOYdW4NSR/0zJmVmymK8QQggxSUiAizd2d/8YOFekAjfyMiIQDnDIYr5CCCHEZCABLt7YnObJ7DW7HU0PMprVXtLy8gFor495MgohhBBCnEIkwMUbm6u/AqdpaFoQ1JErcKk5udgcTlqqD493C4UQQggxwSTAxRubyxwDB4CmAzaM9YyHZ7FYySgolAAnhBBCTAIS4OJN1CxUAM2ig2ZBD40c4ACypk6nVQKcEEIIccqTABdvomahQjjAAUF/6Ig3zZo6nZ72Nvp6useteUIIIYSYeBLg4k3ULFQAzWpU3rxe/xFvmjV1OoBU4YQQQohTnAS4eBM1CxXAagkA0NF95KpaZjjAtVRXjU/bhBBCCBEXJMDFm6hZqAB2m3Fmhe6u3iPeNDkzC4c7QSYyCCGEEKc4CXDxZtAYOLvdGAPX3X3kAKdpmkxkEEIIISYBCXDxxjZwDJzDaVx6ur3D3GCgrKnTaampOuKyI0IIIYQ4eUmAizc254AA53RqAHh6RhfgMqdOx9vdRW9nx3i0TgghhBBxQAJcvImMgQtX0JxuJ5oewus58ixU6J+J2lIl3ahCCCHEqUoCXLyxu4zLkBHY7IkJ2IIevD3BUd08a+o0AJnIIIQQQpzCJMDFG1s4wAWMszHY3UnYAx4CvUdeyBcgITUNd0qqBDghhBDiFCYBLt7YwrMWwjNRHQlGgAv26qM+RNbU6bRUVY5H64QQQggRByTAxRub27gMT2RwJiRjD3rQj7yKiCl/7nyaDh3E33cUNxJCCCHESUMCXLwxK3BGgLMmJWEP9KK8o/9TTVu0BD0Uombf3vFooRBCCCEmmAS4eBMZAxcJcCmp2AMelN826kNMKVqA1Wajas+u8WihEEIIISaYBLh4E5mFGh4DZ01NwRbsQdOtBPyjm8hgdziZMm8BVXt2jlcrhRBCCDGBJMDFm0GzUK0pKdgDHgC8PYFRH2Za8RKaDx2kr7trzJsohBBCiIklAS7e2AZW4CwpKdgDxmQEX+9RBLjFSwCo3ivdqEIIIcSpRgJcvBk8iSElBXvw6CtwubPmYne5pRtVCCGEOAUdMcBpmmbRNO2sE9EYQdQyIuEKXFIStkiA84zubAwAVpuNqQuLZSKDEEIIcQo6YoBTSunAfSegLQKiKnDGGDjNYkFZjdNqeT2jr8ABTF10Gu31tXS3toxpE4UQQggxsUbbhbpB07QPaZqmjWtrxJAxcACa06i8HW2Am1ZsjIOr3LFlbNomhBBCiLgw2gD3aeAfgF/TtC5N07o1TZPpjePBPnAdOACVYAPdd1Rj4ACyp88kY0ohe1/bMJYtFEIIIcQEG1WAU0olK6UsSim7Uiol/HPKeDduUjKXEekPcHqSG0uolz6Pb5gbxaZpGsUXXkLd/lJaa6rHspVCCCGEmECjnoWqadrVmqb9PPx11Xg2alKz2ECzDKzAJSdiD3rwdHtHuGFsi867EIvVyu5XXxzLVgohhBBiAo0qwGmadhfwJaAk/PUlTdN+Mp4Nm7Q0zZiJGhXgtOQk7AEPfR7/UR8uITWN2StWU7LxFULBo+uCFUIIIUR8Gm0F7grgYqXUn5VSfwYuA64cv2ZNcjbngABnSU7G6ffgO4plRKIVX3gxfV2dHNz63li1UAghhBAT6GgW8k2L+j51jNshotlcAwKcLTUVp9+D3zO6c6EONmPJcpIyMtn9ygtj1UIhhBBCTKDRBrg7ge2apj2oadp6YCvw4/Fr1iRncw5YRsSWmo494CHQp6OUOurDWSxWFl94KZU7tlJfUTaWLRVCCCHEBBjVmRgAHTgDeAL4J3CmUurv49y2ycvuNk9mD+AMBziUht97bFW4lVe9n4TUNF77y/3HFAKFEEIIET9GeyaGbyil6pVST4e/Gk5A2yavQRU4Z3om9qBxQvujXQsuwuFO4Owb/ou6shLK33lrTJophBBCiIkx2i7UlzVN+5qmaVM1TcuIfI1ryyazQWPg3OlZ2AM9wNGfjSFa8ZqLyZo2g40PP0AwIDNShRBCiJPVaAPcDcDngI0Y49+2AnJ+pvFicw2owCVk5BhdqBxfgLNYrJz/0VvobGrknX9JD7gQQghxshrtGLhvKaVmDvqadQLaNznZXObJ7AHcaVnYwl2ovuMIcAAzTlvGwvMuZPMTf+fgdllWRAghhDgZjXYM3NdPQFtExJBZqKljUoGLuOiTnyV72gye+83P6WioP+7jCSGEEOLEkjFw8cg+6EwMdju6ZlTkjnUSw4DDO11c/dXvoqHx1C9+TF9P93EfUwghhBAnjoyBi0c254CT2QN4XaApL97eYzsbw2BpuXlc+aVv0F5Xw2O3fwtPR/uYHFcIIYQQ429UAS7G+DcZAzeeBk1iAPAl2LHqnjGpwEXMWLKcD3zzdjqaGvj77d+kq7lpzI4thBBCiPEzYoDTNO0bUd9fN+i6O8erUZPeoGVEAPwJdqzBHvq6j/6E9iOZftpSrv3uj+jt7OSv3/oSB7a+M6bHF0IIIcTYO1IF7sNR33970HWXjXFbRITNBSEf6Lq5KZjoxB7ooW8MK3ARBfMXcNOdvyQlK4cnf/pDXn3wDwS83iPfUAghhBAT4kgBThvm+1g/i7FicxqXof5u1FCiC4evG+8YV+Ai0vML+MiPfs6yy9/Htuef5oGvfobyd9+W024JIYQQcehIAU4N832sn8VYsbmMy6huVD3JjcvbRV9PYNxClc1u58KbP80Nt9+FMyGRp39xJ4//6LvUlpWOy/0JIYQQ4tjYjnD9Ek3TujCqbe7w94R/do1ryyYze/hXG/CC2/hWJSeS0NeNHlL4+4I4E+zjdveFC4r56F2/ZseLz/LOvx7j0e9/nRlLlrPyqg8ybfESNE2Kr0IIIcREGjHAKaWsJ6ohIkqMCpyWnGSeD7WvOzCuAQ7AYrWy/PKrWbzmEra/8AxbnvkXj//4e2QWTmPppVex8Lw1OFzucW2DEEIIIWIb7Tpw4kSKjIGLWkrEkpKCw28suDvWM1FHYne5WHXNtdx63wNc9tnbsDkcbLj/t/zhMzfz6vo/0lJ16IS1RQghhBCGI3WhiolgC1e2oipwtpQUHJEK3DjMRD1ikxwOFp2/loXnXUh9+T62Pf9vdrzwDNuee4rsGbNYeO4aFpxzAYlp6Se8bUIIIcRkIwEuHpkVuKgAl5qOPXDiK3CDaZrGlHkLmDJvAb2dHex7eyMlG1/l9b/ez8aHHmD6aUuZd+Y5zF6xmoSU1AlrpxBCCHEqkwAXj2KMgXOkpQ8YAxcPElLTWH751Sy//Gpaa6opffNVSt98jRd/fy+aZqFwwSLmrDqTOaefSUpW9kQ3VwghhDhlSICLR5FZqFFj4FzpWVj1IFZLiL6eiavADSezcCrnfPhjnH3DR2mqPED5u5uoeG8Trz74B1598A/kzprL3FVnMuf0M8gomCozWYUQQojjIAEuHkUqcIE+c5M7LdO4yuKLmwpcLJqmkTtrDrmz5nDOhz9KW10tFe9touLdTbz56F9489G/kJyVzcwlK5ixdDnTipfiTEiY6GYLIYQQJxUJcPHINrQC505Kx2sBG30TOgbuaGVMKWDVNdey6ppr6W5r4eDW9zi0cxv73n6dXRv+g8VqZcq8BcxYspwZS1eQM30mmkUmRwshhBAjkQAXj2JMYkhyJNHiAqveOyGzUMdCckYWSy6+nCUXX04oGKR+/z4qd27l0I5tZnXOnZxC4cJipi46jakLF5NZOE26W4UQQohBJMDFoxjLiCTYE+h1gS0wfudDPZGsNhuFC4spXFjMuR9Zh6ejnUM7t1G9dxdVe3dR/s7bALhTUpm6cLEZ6DIKCiXQCSGEmPQkwMWjGBW4BFsCPS6w+zvpCJ8P9VQKMolp6Sw6fy2Lzl+LUoqu5kaq9u6iZu9uqkp2s3/zmwC4klOYMq+IKfMWUDBvAbmz52B3ylndhBBCTC4S4OJRjDFwdqudPpcFl7cL3T3+50OdSJqmkZqTx+KcPBavuQSlFB2N9dSU7KG2rIS6/fs4uPVdwDjlV86MWUyZt4D8eUXkzZpLam7eKRVuhRBCiMEkwMUjqw0stgGzUAF8CTZSezsg/cScDzVeaJpGet4U0vOmsPjCSwDo7eqkvryMuv2l1O0vZdeGF9j2/NMAuBKTyJk5m9zZc8mdOYe82XNIyc6VUCeEEOKUIQEuXtkTIdA7YJM/wYGzrR0wTqeVljsRDYsPCSmpzF6xitkrVgEQCgZpqTpEY2UFjQcqaKysYOszT6KHggC4kpKN5U1mziZ31hyyps0kLS8Pi8U6kQ9DCCGEOCYS4OKVMwl8PQM26UluEno6gIk9nVY8stps5vpzrDW2BQMBWqsP03Cg3Ah2ByvY8sy/0EMhAGwOJ5mFU8maOoOsadPJmjqdrGkzSExLl2qdEEKIuCYBLl45EsHfPWCTlpyM09cGSIAbDZvd3h/qwiKhrrnqEC1Vh2ipPsyhnVvZ+/rL5j6u5BSyw2Euo2AqGVMKSJ9SQFJ6pgQ7IYQQcUECXLxyDK3AWVNTsAeqAE7ateAmWqxQB8aYupaqw7RUh4Nd1WH2vPoSAV//TGC7y016/hQyphSSMaWQ9CkFxmX+FJkJK4QQ4oSSABevnEng9wzYZE9Nw6oHsTs0qcCNsYSUVKYVn8a04tPMbUopetpaaaurob2ulra6GtrqaqjbX8q+tzeCUua+SRmZpObkkZabR2puHmk5xmVqTh4JqWlSuRNCCDGmJMDFK0cy9B4esMmZbpwP1ekkrs+HeqrQNI3kzCySM7OYvnjpgOsCfh8d9XW01dXSVldNZ2MDHY0NHN69g57XWwfsa3e6zDCXmp1jHDMrcplFYlq6TKYQQghxVCTAxStHIvgGjoFLSM8GwG4N4O2RCtxEsjucZE+fSfb0mUOuC/r9dDY3mqGus8n46mioo2rPTgLegcvDWKxWkjIyw2Exm+Ss7P7vM7NIzsjEnZwi54gVQghhkgAXr5xJ4B84Bi4pw1g3xKr10SsVuLhlczjILJhKZsHUIdcppfD1euhuaaa7tYXu1ma6or6vL9/H/s1vmcufRGgWC4mpaSSmZ5CYlt5/mZZBYno6SWnGzwlp6djsk2N9QCGEmMwkwMWrGJMYkjPyAdB0j4yBO0lpmoYrMQlXYlLM6h2A0nV6uzrpammiu7UFT3sbno52esKX3W2tNBwop7erc8A4vAhXUjLulFTcySkkpKTgTg5/mdtSwz8b2+xOl4zRE0KIk4wEuHjlTIaQD0IBsBoVlfTMAryAJdhFn/fUOx+qMGgWS7i6lk7+nPnD7qeHQvR2dZoBz9PRbnzf2U5vVxfe7k46Ghuor9hPX1enuf7dYFa73Qx37uQUXEnJuBITcSUm4QyHzf7L8PakJJwJCTJ2TwghJogEuHjlSDQu/T3gTgcgPSGDAy7Q/B3ooVP7fKjiyCxWK0npGSSlZxxxX6UU/r5eers66evqoq+7i76uTvq6u4xtkZ+7uuhuacbr6cHn6Rk29EU43Am4kpJwJiSaAc/hcmN3J+B0u3G4E7C73DgTEsLb3TjdCTjcbnO73eWSICiEEEdJAly8ciQZl77+AJfqSMXjAou3A6yT63yo4vhomoYzIRFnQiLpeVNGdRulFEGfzwxzXk8PXo8HX9TPPo8nvN34vqOxAX9fH35vH/7e3iFj+YZjczrNYGeEPhd2hxOb04nd6cLudGJzOLG7XMZleLvN6cTuiPo+/GVsd2F3ObFYbVKpFkKcciTAxStnOMBFTWSwWqz4XFasvS2QDL1dftJyEyaogeJUp2maEaRcLpIzs47pGMFAAH9frxHq+nqNL29f1M9Rl+HQ5/f2EfB66e3qIuj3EfB5Cfh8BH0+An5fzHF/Iz4OiwW704nV7sBmd2C128KXDmwOO1abHZvDgdUe/b0Dm92O1W4P72sP/+zo3zdync2O1WbDYrNitdmxWK3Gz1ZbeHv40mpcr1ksEiiFEMdtwgKcpmlWYAtQq5S6StO0mcCjQCawFfioUsqvaZoT+AuwAmgFblBKHQof49vALUAI+KJS6oUT/0jGiSPZuBy0mK8v0YGjxwhwnk7fBDRMiNGz2e3Y7KkkpKSOyfGUUgQDfiPMRQc7n9cMeP3bwtv9PgJeL6FggGAgQCgQIOj3Ewr4jW1+P15PD6HIdQG/uV8o4D9iN/KxGBzwLDYbVqsVi80evrRhtUbCn7E9OhharFYsFguaJXxptWKxWvq3RV9vtaJZLOFtUd9bLcb3I+zTf33kWJb+6y3G7dE0NM2CZtGMbVp4m0Uz9tW08M9GcDX3sRi3s0QCraZJsBXiKExkBe5LQCmQEv75buBXSqlHNU37PUYw+134sl0pNUfTtA+H97tB07SFwIeBRcAU4GVN0+Yppcb+1XYiRMbADVoLLpTowtnSDPnQ2ykzUcXkomma0WXqcOJOTjnyDcaArofCwS5AyB8JfUbQCwX8RtALhdCDQUKhIHrQ+AoFg+ihEKFgIHwZHLDPsNcHg+ih/uuDgSC612ter4eC6LqOHgqhwpe6rqP0EHpIR4V/1vXQUVcrJ1w4xA0OeZFthMPe0J/7AyRa/z5aOGRqaKARDogaxoXWHxyJBEjC10f2t5jbogNmdOA0Q6f5fdRxLP3HNu6HgcexWAZtG9RO8/ZEXR/7+NHtNNs45Nfb31ZzW9Tvfsh+aIOvivohxnXEuO/IXRLjvge3caTrYrR/4F0Pvm9t8FVD2h77uqFtHe4xalYLi9dcMqSdJ8qEBDhN0wqBK4EfA1/RjN/4hcCN4V3WA7djBLhrwt8DPA78X3j/a4BHlVI+oFLTtApgFbDpBD2M8RWjCxVAJSXgrmzEarPg6ZAKnBDjzWKxYnFaT8rz3SpdN8OcGfYiwU8PoUJ6VBgM9X8fFQIj36tQCF3pZkhUSkcphdJ140up/p/NSx2lq5H3NX8e5T561LGV6j/+gPvt36bruhlklVKgFMr4wbwP43oVvlAoFCgjvPffJryfwvzeuEofsG2440e2GffTfxylD90WfRyl6zHbPuD4UW2H8DHMJ0HkMtZ1KvpiwA1UZOOAQyli3GDSsjmcky/AAfcA3wDC/YRkAh1KqciI5xqgIPx9AVANoJQKaprWGd6/ANgcdczo25z8oicxRNFSknD31ZKQ6sDTJQFOCDE8zWLBarFgleHOYpz1B77+cNcf+AbtE7VRDQqYA283NIjGuq7/20Hh84i3ixFEVYxjDN4/TgLsCf+v1jTtKqBJKbVV07QLTtB93grcCjBt2rQTcZfHzxkZAzcwwFlTUnEEISHFjqdDulCFEEJMvBG7Z8W4mIiTK54NXK1p2iGMSQsXAr8G0jRNiwTKQqA2/H0tMBUgfH0qxmQGc3uM2wyglPqDUmqlUmpldnb22D6a8eKI3YVqT00zrnaF6JVJDEIIIcSkdMIDnFLq20qpQqXUDIxJCK8opW4CXgWuDe+2Dngq/P3T4Z8JX/+KMuqYTwMf1jTNGZ7BOhd49wQ9jPFnd4NmGdKF6krLBMBi8+KRSQxCCCHEpBRPAyO+CTyqadqPgO3A/eHt9wN/DU9SaMMIfSil9mqa9hhQAgSBz50yM1DBKEM7hp7Q3p0eqSB24+9LJOALYXfKKvZCCCHEZDKhAU4p9RrwWvj7gxizSAfv4wWuG+b2P8aYyXpqinVC+8w8AFSoHUjE0+kjLUcW8xVCCCEmk4kYAydGyzm0ApecmQ+ACrQBshacEEIIMRlJgItnjsQhAS49qxCAkLcRkLMxCCGEEJORBLh4FqMLNTE8Bk55wgFOFvMVQgghJh0JcPHMmTykAmdxOvHZNVRXC1abRbpQhRBCiElIAlw8izELFcCbYAVPD4lpDulCFUIIISYhCXDxzJE4pAsVIJDgwNLTS0KKUwKcEEIIMQlJgItnMWahAoQSXdh6fCSmOaQLVQghhJiEJMDFM0cyBL0QCg7YrCcnYO8LkJjqlEkMQgghxCQkAS6eOWOfD9WSnIy7L4QrxYbfGyLgO3VOQCGEEEKII5MAF88cicbloABnTUkhyQsqIQDIWnBCCCHEZCMBLp45whW4QRMZnGkZJPigx9oOQK8EOCGEEGJSkQAXz5zJxuWgClzhlCIAdre8DYCnQyYyCCGEEJOJBLh4ZlbgugdsTsjIAeC92pcB6UIVQgghJhsJcPHMHAPnGbDZlpkJgK+tCosNPLKUiBBCCDGpSICLZ8N0oTpmzACgsMNC0O2TpUSEEEKISUYCXDwbpgvVPmUKmt3OSv8U2qyNdLd5J6BxQgghhJgoEuDi2TDrwGlWK/bp05jXnUSTo5qWui6UUhPQQCGEEEJMBAlw8cyeAGhDxsCB0Y2a1tRLZ2IzwT5Fb1f/OLiGyk4qdzafwIYKIYQQ4kSSABfPNM3oRo1xQnvnzJkEq2vIzE0AoK2+P+RteuIA//njHrpa+k5YU4UQQghx4kiAi3fOJPB3D9nsmDEDAgFyHcZ5UtvqjACn64qmqm70oOKdpw+eyJYKIYQQ4gSRABfvhqnAOWbOBKDQE8Rr85gBrr3BQ9AXIi03gf3vNtJcNTT8CSGEEOLkJgEu3jmThh0DB5DbFqLNXU9zbScATYeMwLb25gW4kuy89c8KmeAghBBCnGIkwMU7R9KQWagA1vR0LKmppDf20p7QQHt9L0opmg93YXdayZ2ewulXzqC2rJ26/R0nvt1CCCGEGDcS4OLdMF2omqbhmDGdhIYO2twNBL2K3k4/jYe7yZ6WjGbRmL86D4Am6UYVQgghTikS4OLdMJMYAJwzZmKraaI9oR6A5upuWmq6yZmRAoDDbcPutMqZGoQQQohTjAS4eDdMBQ7AMXMGqqkFj80IcBVbmtCDipzpxim4NE0jKd1JT7sEOCGEEOJUIgEu3jkSY05iAHDMCM9E7dPRnQEObG8CIDdcgQNITHPi6ZBTbQkhhBCnEglw8c6ZDME+CAWHXOWYOQOAuT1J+FK6CPp1XIl2kjNd5j5JaVKBE0IIIU41EuDinSP2+VABHNOnAzC9w0ZnolF9y5mejKZp5j6J6U48nX50XZYSEUIIIU4VEuDinSvVuPR2DLnK4nJhm5JPfpuiyVUNYE5giEhKd6F0RV/UuVKFEEIIcXKTABfvEjKMy962mFc7Z80mq6GPQ459AOTNTh1wfVKaE0C6UYUQQohTiAS4eOcOB7i+YQLc/Hkk1XbQ6Krgsq8WMW1hxoDrE9PDAS7OJjJ0tfbh6w1MdDOEEEKIk5IEuHhnVuDaY17tKirCEgwxpQ30LM+A8W8ASenxWYF78pfb2fTkwYluhhBCCHFSkgAX745YgZsPwPRGRXNv85DrXYl2rDZLXC3m6+sN0N3qpb0+9vIoQgghhBiZBLh45043LocbAzdzJtjtTG9SNPU1Dble0zQS0xxxVYFrb+wFoKulb4JbIoTQQzpKySx1IU42EuDindVmzEQdpgKn2e04Zs9iehMxK3BgzESNpwpcRzjA9XT4CAX0cbufnnYf21+skjcncVy8nsAp+xwKBXTWf/ttSt+un+imCCGOkgS4k4E7Y9gKHIC7aAEzmqG5L3aAS0xz0tMeP5MYOhqMAIeC7rbxa9fb/yzn7ScqaI/c3yjoIX1cQ6U4ufR2+XnwW2+x/52GI+77+N1b2PL8ofFv1Bhqreuht8tPS1Xs8y0LIeKXBLiTQUIG9LYOe7Vz/nzSehTdDTUxr09Kd9LT4RuxijDSdT3tXp797S68PWMzazRSgQPoHKdu1PYGD+VbjS7lzubR3YdSiv/8YQ//uHvLKVtxEUenvqKDUECnctfw/38Afm+Qxsoumg+fXEGopdpYINzTKetEiqGqS9tG/fopTjwJcCcDd8awXagAriJjIoO10ljM94ebfsgXNnzBvD4p3YkeVMMGsObqbh76n0289c+KmNcf3NHCoV0tVO8bvg1Ho72xl+xpyQB0xwhwfd1+/vOHPfR1H/ubypbnDmG1Gk/vzqbRVeAqtjZRubOF1poemqUiIYCGg50A1O5vR41wNpPIh5LervgYqtDd5uXBb71FW93IE4Uiz/OeOBpiIeKD0hXP/3437z1bOdFNEcOQAHcySMgYdhkR6J+JmniohX1t+3hs/2O8VvMajZ5GY/sIi/ke3tPKv36+jZ52HzteqooZ0hoPGW9irTVDT+d1tHRd0dnUR8G8NKw2C50tQ7tQD2xv5sC2JmrKhn/MI+lo7KX8vUYWX1CAM8E2qk+QXk+AN/6+n8yCJCxWjfL3Go/pvkX86Wrp4+0nKggFj75rvOFgF2jg7QnQWjf88z/STd97As948u/f7OTdf8deiqfpUBeeDp/5vzuc5mojwI3HGFm/N8j6b7/FwR2xh3acKFueO0Tt/mN7LZnMPJ0+Ar7QET8EiIkjAe5kcIQKnC0jA29aApm1Xdyz9R7cNjcAr1a/CkBSmnFy+8in7KbDXWx5rpJnf7uLZ3+7i9QcNzfevpq03ARe+Usp/r7ggOM3VnYB0FJ7/AGuu9VLKKiTnpdISpYr5kzUmnCIPNbS/dbnD2G1WVh2yXRSs92jOs6mfx3A6wmydt0Cpi3KpPy9Rjl/7AkwUlVrrGx/sYrtL1ax5/Xao7pdKKjTXNXNnBU5ANTsGz4EmBW4Tv8J6X5XSlG7v526io6Y10eGJnS3Dj/GVNeV8aFMM4LnsTzfq/a2DhvQ2uo99LT7OLSr5aiPO1Z8vQHeefogOzdUH9Ptu1r6aKgcOQSfqiLP6fZ6zwn5PxVHTwLcySAhwziZfXD4T/e+mflMbdR5q+4tPrPkM8xImcErVa8A/Yv5etq9tNb18PhdW3jn6Uo6GntZdM4UPvDV5aRmJ7B23QI87T7eeqK/K9XbE6CzqQ+0sanARV4U0nITSM50DwlwSlfUlnUAxxbgQkGd/VsaKTorn4QUB6k5CUfsQm2r81DyZh1L1k4le1oy81bl4un0U1/ecdT3L0YvFNT5+4/fZfNTB8b1Psq3GtXULc8dOqqzfzRXdxMK6sxZnkNqjpvaESrCkTUNgwGdgDd0fI0ehd4uP6GAPmxA6wr/73SPsHxQR0MvwYBO7owU43zJxzBk4Z2nD/LmP8pjXhf5nUS6oSdC46Eusw3Rwbpia9Ow4Tfaqw/t48lfbh8wbney6GgynkPBgE5Xq4yDi0cS4E4GCSMv5gvA3BkUtsAURw4fKfoIF067kPca3qPL34U7xYFm0ejp8PHOUwexO63cfPfZ3HTHGZx/43wcLhsAebNSWXLRNEreqKMt/OIbeQGcXpxJT7sv5ji6Lc9V8q9fbEMPHbmLKvJCmJ6XQGqWi67mvgEvrC21PXg9xn10HUOAa6vzoAcVBfOM9fNSs91m1W84h/cYA9SXXDgVgBmnZWFzWtm/ZWg3anVp23GNzZsI3p4Aj/7wXQ7vHXkg/om25/VaWms95u9/tKpL2kbdHVq1txWfJ8jqq2fi7Q2w7YXDo76fxoPGcz9vViqFRRnUlncM+xxvb+wlchIUT+f4jyeLBLeeNl/MNkU+GPWMMMs70n06Y3EWcPTdqEop2ht76W71xvyfaKsPV3AaeifstHmR3oO+7oD5gTAU1Hnlr6W8+++Rx3b1dvmpLWsnFNB55S+lY16R727z0jHK8bkToSNq9n7kbyniiwS4k4F75BPaA6QuWIxNhy9lfgiXzcWF0y4kqIJsrNmIxaKRmOrg0K4WKne2sPTiaSSmOmMeZ+lFU9E02P+usWxCY2UnmgYLzsoHoKVm6OD+iq3N1JV3sHsUXVTtjb04E2y4kuykZLvxe0P4evu7bCPdVFMXpI968kG0psPGC3ZkkkRqjhulRu5Kqt7XRnpeglmptDuszFqSxYGtTQOCQl+3n6fv3cHzv989qrAaL/a/10hrbQ+vPbyPgG/8qkOdzb2jnuji6w3w3nOVaBq01noI+EfXrvYGD0/fu2PUYxT3v9uIK8nOskunM39VHjs31HBgWxOv/62Mx+/eMuJi0g0HO0nKcJKY5qRwfjoBb4imGLNM9ZBOR1MvWVON59yRxsGVvl1/zF16EZGKiK6rmDNII2NLR3reN1d3Y7VbKCwyPuwcbYDr7fSb1cbIB71o7fUeNIs27PWjvp8uP0/ds/2YlhxqONiF3WUNf99pXga8IdpGGNMIcGBbE0rBsounUX+gk92vxZ7lf6xef6SMJ36+bVz/J49HR1MvKVnG8Jsj/a5Gq7W2h63/OTQur5/VpW1sfHT/CfkAFS8kwJ0MRlGBm3PB1WC3seRFY1Dz4qzFZLuzB3SjttZ6cCfbWbJ26rDHSUx1Urggg/L3GlFK0VjZRcaUJPJnpwHQMqgb1d8XpK2uB4tF492nDx7xzauj0UNabgKappGSaYzVi34TrdnXTlpuAlPmpuHp9BMc5Rt7RFNVN84Em/nCk5qdYNzvMGEwGAhRX97B1AUZA7bPPT0XX2+Q6pL+33lNWTsoqD/QybYXqgbs39Xax8sPlnD/V99gw19KzSA5WkopSt+uH5cXn7J3GnCnOOhp88Vcp8zfF4wZzEdSU9ZO1aCK3ttPHODpe3aw7cXYVS6vJ2D+Pbc+fxhfb5DTr5qJ0tWoZ/1GBlS3j6JLy9cXpHJXC3NX5mK1Wlh9zSwA/vOHPezbVE9LTQ+vP1I27Ji1hoOd5M1KBaBgXhpAzIk1XS3ecNXX2OdI/wNbnz/EjperRtznSLqiJv90D+re0kM6Pa1e0IyJS8ONX2qp7iZzSiLJGcb/ytEGuPaG/sHtTTECWlu9h2kLM0ALTwY5RrVl7dTsa6dy59GNpTNevzqZszwHh8tKwwEjwFWF/6f7ugMj/q0qtjaRnp/ImR+czbRFmWx+8sCYLqnRXu+hr8vPrleHhnm/N8ijP3yHyp2jnwDS1+3nzcfKxywQdjT2kj0thaR0p9kjczyUUrz60D42P3mQVx/aN+bj6na9WsPu12p45I532LepflIsBSUB7mQwigqcPT+frE/dStczz+LZtAmLZmHN1DW8Wfsm3qCXxPBEhhWXzzC7TIcz7/Rculq8NBzsovFQF7kzU0hIcZCQ4hgyDq7xUBdKwdnXzSEY0Nn0rwo6mnp587Fy/vOHPUPemDsaeknPNUJVSrbRpsibUSioU1fRQWFROinZRriLtU5cR1Mv+zbV09cz9MW3+XA32dOS0cL9WamR44THcyilBoxnaTjQSTCgUzgowE1dmIEzwcaBqAHaNfvacbiszFmRw3vPVNJY2UX9gU42PlLGwz/YTMWWJvLnpFKxtYl//GQLf/rKRv7ynbf5x11bjvjm2FjZxSt/KWXbf0bfxTca7Q0emg51sfySaRSdkceOl6oGvPF6ewI88fNt/OPOLQPezDqb+3j1r6XDvsFtfKSMjX/fP2BbS00PFpvGpicO8M6/Dw54AdVDOg//YDP3f/UNnvm/nex8tZqi1XksOrcAiB0AYj6eozgN28HtTYQCOvNW5QKQnOHiys+exqWfKuYTPz+XM98/m6q9bRzYNvRNsqfdS0+7j7yZRoBzJzvILEyKOQ4u0qaC+UYlq3eENdU8HT46m/voaffh9waH3e9IusMBzfx+QNt96LoiqzCJUFCnL8awB6UULdU9ZE9LNoZYaEe/lEhk5q072U7joYH/5wFfiO42L7kzU8ickkjjcUwEiMz+PdqxdJ1Nffh6g+TNTiVvVqp5+6q9rdicRlWudZiJWZ5OH3UVHcxZkYOmaaz5r/lomsZbj8ce73ckoaA+oJofCurm33D7i1VDupj3v9NAa62H8i1DT484nIqtTex8pZqD20d/m5Ha29XqJS3XTUZ+4pjMRK0r76Cxsou8WSns29TAG4+VH3PIeufpg7z8YMmAbS3V3RTMSyMjP5EN60spebPuuNsc7yTAnQwiFbgRFvMFyPz0rdinTaPhjv9F9/u5cNqF9AX7ePbgs+TPTiWzMIni8BvmSGYtzcZqs/Des5X4eoPkzkwBIKswachM1IaDnaDB/DPyWXrRVPZtauDhH2xm92s11Oxr47GfvMeGvxhBwN8XxNPpJy0vHOAGVeAaD3UR9IUoLEo3K2eR4BVt0xMH2LC+lAe/8Rb//s1O8/ahoE5rXY/ZfQrGm4vDZTU/OR/c0czDP9hMZXhmXPW+djSLZlZPIqxWC9OLMzm0s8Us99fsa2PKvHTOv3E+CakOHv/pFp742VZK3qpn3qo8bvrfM7jiM6dx811nc/5H5jFvdR55s1NpOtR1xE/SJW8ZLzYHtjeP6SfTss0NaJpRUTzzg3OwO6289OcSqva24u0J8PS9O2it60HXFXVRkzb2vlFLyVv1PHvfziFBw9Ppo72hl87mPvM6vzdIV3MfKy6dTtFZ+Wx59hAVW/vfSNobe/H2BMidlUp7gweb3cqqq2eRkOIgOcNljlU6kkj4Hs34yLJ3GknJdpvPXzCC+ZwVOdidVhZfUEDW1CTefGz/kJnXDVHj3yIKi9Kpr+gc8vuIBOK8WalYrNqIVZ3o33Gs53a0jsZe3nq8nNcfKePVh/cN+DDU3dpH5pQkALoGBbjIh57IONDorke/N4hSiu5WL77eIFlTk7FYNBJSHEe9mG97Yy92l5UZi7NoOtQ14M24o7EXFKTnJZI7M5XGyq5jfl5HwsPRBrjI7NHcmSnkzU6ltc5DR2MvLdU9LDp7CjB8gDuwrQkU5gzkpHQXyy+bTuXOlhEnswymlGLf5noe/OZbvPbQPnN7d6sXpWDJmqn4eoNsf6lqwG32bDSGo9Ttbx91yIn8Dx3YfvzLtnS19KF0RXpuAulTEmlv6DXHAD7zfzuHfHjrafce8UPVthcO4062c82Xl7H0oqnsfq2G+7/2Bn/5zts8dud7bPrXAeorOqgqaeXtf1bwn/+3O+YMYKUr9r5ZR/mWRvOsOX09fnrafUwvzjIm5eW4zdf4U9nIpRgRH9yjmMQAWJxO8r7/fao/+Ula//QnVv33p1ies5wfbf4R96y5hxsuPM+sTI3E4bYx47RMszIReQPMLEyi5pVqQiHdXCS34WAXGfmJON02Vlw+g86mPjKmJLLovAJsdgtbnj/MrleqObSzhUXnGS+aaeEKnMNtw5VoN//xa8vaQet/44GhlRZdN5ZPmLE4k4wpiex6tYZ3n6nkopsXmhMYogOcpmnGTNRm442//F1j7NSmJyqYviiDmtI28mamxKxKzlySzf53G42xUOkuulq8nHbhVFyJdi69tZjdr9UwfVEmMxZn4XD3397ptlF8fiFgvBjXV3RQU9ZubgNjDJgzwQ4Yb6oVW5qMN9EOH42HugYEh6PR3ebl+d/vZvEFBRSdkU/ZOw1MXZhpjnk8/8b5vP5IGf/+zU4sVg00uOK/F/PSn0uo3d9uvmFVl7aRlO6kubqH//xhD1d+9jSsNuNvXre/w7gzZYxfy5+danaxZE1N5vQrZ1K5o5masnbmrjSqX5EV/8+9fi4ZUxJRusISfg7lzEgZEODe+Pt+bA4rZ35g9pDHFwlwRzqDh6fTR+3+dlZeMWPY57zFauH8G+fzz59u5el7d5A7IwV3soO+Hj+1ZR1Y7RaypiaZ+89ems3Ol6s5tKuFeavyzO3tDb24Uxy4Eu0kpDhGXMy3NirAtTd4BjxXB3v33wep2NqEM8GOry9IKKBz0c0LAaNqnTU1GW+Pf0gFLhJuC+ans3NDNd2tXnJnpODp9PHQ9zaRlpdgDomI3H9imjNmlbiuvJ2DO1tYdeXMAc9xMLoA03MTyJmRQunb9XS1eM2Kd+T5kJGfSMAXpOTNOjqaeknPS4z5WJVS9LT7aK3pwdcbYN7qPPPv1lrnQdOM0OPp8JGY5kQpxeE9rRQWpWOzW2Mes+FgFw6XlYy8RHpn+UEZM5EB5q3OZf+WxmEDXMXWJjILEsnI72/v0rVT2buxljcfL+f6b59uju8brKfd6L3oafdyaHcrtWXtWCzagFmvkQ+Us5dn4+nysfOVGhZfUEhiqpOGA5201nrImZFC06EuOpv6zNfMkUTGGVaVtBHwhbA7Y/9eBmup6WHf2/Uc2NHEGdfMZv7qPPP/LDU3gVBIhWc89xEKGL93h8vK2R+aY74mPP//9tDX5ee/fniG+X8drbm6m6q9bZzx/lnYHFbO+tAcUrPdtNZ5CPpCdLV62f5SlTnJyGLVsLusHNzRzJK1U1l19SzsDqvZ3r7wh6Tm6m7yZqXSUmX8HbOmJaFZNArnpxtLQYX0mO05VUiAOxk4EsDmGrELNSLpnLNJWruW9r8+RNanP819a+/jUy9+itteu43/W/t/nDXlrFHd5bxVeRzY1ozdZTVfdLMKk9CDio6GXjILklC6McZk9rJso5kuG5d9evGA45z9oTksOCufDQ+WsPV5458z+sUoshZcMBCiYmsT2VOTcSUawcaZYBtSpWiu6sbXG2Teqjzmnp6L1xNk/3uNnPfhoDnuLGf6wDfF1Gw3zVXdBHwhDu9pJT0/kfZ6DzterqapqpvTr5wZ83cwbVEGVpuFg9tbyJhi/A6mFhlhOm9mqtm9NhJNM15MDu1uRekKzaLRWtvDYz95j5WXz+D0K2dyYFsTAV+ISz65iOd/v5uKbU3HHOAqtjTRXNXNK3/ZR9k7DfS0+zjrg3PM6+euzGXWkmwqd7VwYFsT88/IY8biLPLnpJqVhb5uPy3VPay+ehYJqQ5e/es+Nj1xgHOunwtAzf52LDYNPahoqe42Alyt8YadWZCIZtHImZ48oFu0paYHq81CWp4x/lGz9r/55c5M4cC2JrNKu+u1GlDGm1vO9P7qWaT7W7No+DzBASF4sAPbmgdUUIaTNzOVsz44h9K36ijdVE/AG8LmtJKc7mTZxdPMNygwKmyJaU4qtjYNCHAdDR5zWIAR4EauwBXMT6Nuf8eI5+gN+ENU7m5l4TlTuOCmIp65b6dZgVO6orvNy6xl2Xg63EMqcF0tfVisGvnh51DkPMiNlV0EAzq9nX52v1aDZtHIDD+vE9OcA8Z3eT0B3n6igtK3jJPct1R1c9UXlgwISx2NvRTMSyd3hvE3ajrUNSDAWSwaqTlus6u34WCX+VrS0dTL4T2t1Fd00NncR1dzH/6o5VeSM91MmZtGwBeiq6WP6YsyObynlYaDncxenkN9RSfP3reLVe+bOez/b2NlJzkzUtAsGrkzUszJWe5kO9lTk8kqSKS1dmjX4KHdLdRXdLL66oHHjXyoeOnPJZS900DRmflDbuvp9PHoD981J2Y5E22c9+F59PUEeO+ZSjNYRT5QpuYksPp9s6jc0cJzv93FNbctY8/GWhwuK+d/ZB7/+MkW6so7SMtNQCnFxkf2k56fwOILCgd8MPF6AnQ09jJ1YQbVJW1U7W1l9vKRn/tgVMU2/esAFquGzWFlz+u14QBnPBfSchIgXABsq/NQHx5H6PeGqClrZ/qiTDoae83/9QPbm80PbdG2v3AYu8tK8XlGD5CmaQM+0ILxoba6tB27y8qUOWkoXfH2ExXseLmajqY+rvzsaQBUlfT3REXGqUZmVGeHJxIVzEtn7xt1tNT0DHgNGUt15e3U7u9g2SXThv0QMd5O3Wh6qnFnQN/oSvcpV1xOqL0d7+7dJDmS+P3Fv2dG6gy+tfFb6Gp0s3+mL8rEmWAjd0YKlvAnzcxCoxrREv5n6WjqNbpYjxA2MvIT+dA3VrD66lnMWJw5MMBlu+ls8fLaQ2W01XlYecUM87rUbPeQSktkkd/IeKP5Z+QR9IU4uKOZZnMCg3vAbVKzjTe5yp3NBAM65394HnmzUtn85AFQMLUonVgcLhtTF6RzcGczNfvaSEh1kJ5/5E/CgxUUpeP19K/kX/ZOA3pQ8e4zlRze20rpW/Wk5SYwvTiTqQszwrPfjq276fAeI2yuvGIGtWUdOFxWZi7JGrCP1W5hzoocLv1UsbmERMG8dNobevF0+syB+lMXZLDw7CnMPyOPvW/Vmd2MtWXtTFuYiTPRZj4XWmt7sDmtZrd49vQU2mo9BAPGm3JLdTcZUxLNym20SIW38VAXu16pxmLVcCXZefMfA8fIeHsCxpimWcb+kbGTekjn+d/vHjDhpGJrIxlTEs1uxpEsu3gaN95+Brfecz6f/s353HrPedx4+xmsvnrWgP00i8bs5dlU7W0zfxdKKdobekkPV2oSUp3DBri+bj/t9R6mLsggJcs9YoA7vLuVoC/EnPCbYfa0ZNrrPQR8ITydfvSQIiXTRXKma8gkhs5mLylZbpyJNuwuq9mF2nS4C82icdP/nsEln1zEhR8twhauakRX4JRS/OsX29i3qYFlF09jzUeLqN3fwQt/3GsOJ/B7g/S0+0jLSyCjIBGr3TJgpml7vYfUHDdWm4X03AScCTYaKjs5tLuFR3/4Dg9/fzNvPlZOc1U3ialO5p+Rz3kfnsf7vrgENOPNEcLd0wrmr87DarOY3aj7NhnBsuTNupjLewR8IVpreswPQg63jYyCJJQynteaRSOjIIm2eo95+4AvxOt/K+PZ+3aRMSWRhecMHW4y9/RccmemsPHR/Wz8+/4hk3/efqKCgD/E+29bxi2/OJdbfn4uiy8oJKvAeB5GKpOdTX3YnVbcyXbSchO49FOLaK7u4Zn/20nFtibmn5FvjE9MtlMb/l3UlXewZ2Mtb/y9nBfv3ztgskIkQC27aBquJPuAblSlFL5e4/UnepJUKKCz4+UqCovS+fjd57Dskmk0HOw0lzdxJdlxJdrNKmRrbU+4op+B3Wk1F3CuCK+zmJTuZMdLVUNeuzqbe6nY2kTxeQXDfuACcCbYmbMih+mLMrE7rTjcNi64qYiVV8zg0K4WcyJa1d42sqYmkZzpMp8PLdXdJGe4zA//U8JDYiJrio5Gb9foF+HWdcUbj5UbQ18mcK6EBLiTRULmqCpwAElnnw0WC92vvw5AqjOVdQvX0e5r50DH6BZNtdotXP7fiznnurnmtvTcBKw2izkTNfLPM5pKlMVqYeUVM7jyc0sGvImnZLrpau6j7J0GVr1vJrOWZvdfF+MsCjX72sksSCIhxQFA/uxUUrJc7NvUQNOgCQwRqTlulK7Y/lIV7hQH+XPTOOuDs1EK7C4rOTOH/4Q2c2k23a1eDu5ooXB++qi6oAeLdAnX7DPOp1n+XiOFRelkTknixT/uof5AJwvOykfTNGYvy6GnzUfT4W68ngCv/rWUPa/XjGr8kK8vSH1FJzMWZ7H66ll84KvLuOzTi8036RHbGA7EdeUdVJe24UywkR2uZBafX0DQF2L/e430tHvNU6FlFSabz4XW2h4ypySa3Uo505PDK/17jAHzNT0DuiOjZU9LRrNoVO1tpXRTPfNOz+WMa2ZRX9E5YIJBZLLA9OJMoL8bqr2hl4M7mnn90TJCIZ2edh/1BzqPWH2LxWa3jvg3nrM8h1BQ59BuY3xNX7cRKqMrcMONJYt0oU2Zm056XsKIi8NWbGkkIcXBlLlpAORMS0Ypo5IZWUIkOdNNSqbLWAsu6vnR1dJHSpYLTdNIznCZXazNVd1k5CficNmYuzJ3QAUpMc2JrzdI0B+is7mPtjoPZ187h7M+NIeFZ0/h3BvmcWhXC5ueNGa5R6/naLVayJ46sOLa3tBrvvFHKmClb9Xz7H27CAUV594wl//64Zl89EdncdXnl3Deh+ex+IJCpi3MJHNKkjlWMFIhy56WTM70ZGMJEH+Iim1NJGe46Gn3mbOhlVKUvdNA2TsN7H+3AaUYMP4xUpGctsh4/mROSSIU0M3lil744x72vFHL0ouncd23V5qvMdE0TePiTyxkxuJM9r5Ry99/9B4v3r8XvzdIbVk7+99pZPkl0ymYn44r0W4+lyIV/Mh4vs7mPlJz3Ob1M5dks3bdAuoPdKIHFYvOm4KmaUyZm07d/g5jXNzrtTgTbKx630wObG3inz/das7qbjxknPItd2YKM5dkcWh3C6GATuWuFh74xpv86Stv8Oj/vss/frLFHMN5YEcTfd0Bll1ihL7I/8uBbU10NPZPNnO4bSSlO9mzsY7eTj/F5xYwvTiTyp0t6LqiPDx5a8Vl02k63E19xcBxa9tfqsZitYy4+sFIis8rwGLR2LuxFl9fkIYDnUxblBmemGI855qrB76+JKY6SctNMMPvSJSuePPxch74xps8dud77NlYe8QJRvs21dNS3cNZH5gzqtfX8SIB7mSRkH7EMXAR1rQ03MuW4Xl9o7ltWc4yALY3bR/1XRbMSyezoP+fwmK1kFmQyOG9bcY/0sEunAk20vOOvioVEVnuY/aybFZePmPAdeYivOFP/UF/iPqKTgoX9FfMNE1j/hn51O5vp7WmJ+aYotQco30t1T3MXpaNxaKRPyeNRecVsPDsKTGrQhEzFmeBZkyQKBymUnckyRkucyX/+gOd9LT7WHBWPpf/dzFoGppFY/4ZRpfczCVZWCwaW58/xGM/fo+St+p5/ZH9PPXr7UccJFxd0oauK6YvNt6gpsxNH7I8ynCypyZhd1mpLWunurSNgvnpZuU1d0YKmQVJ7H2j1uxmLZifTtbUJFprPeghndZaj9kdB5jdFk2Hu/B0+PH2BMgqjD3ey+6wklmQyN6NtQT9OkvWTmPB2VPILEji7ScqzCpeJDRE3oCjJ7+AUdUo29QwZAD6WIruRoX+CQyR/4GEFAfe7tinparb34HNbiFnejJpeYl0NPXG3M/vDXJoj9EFFvkbZE8zfp/NVV1mIEvJMipwuq4GjF8zApxRCU1KN0KOUsr4gDM99t8gMkayp8NnhqfIcAGA09YUMvf0XPa+UUvAFzKrh+m5xt88d0YKzVXd6CHdDEXpUePHZi7Nxp1k59wb5vLh76/itDVTze7WwabMTaP+YBehkE5bXQ9Wu4WUbDd5s1Jpquqm/L1GAt4Qaz5ahDvFwd43jAlApW/V8/IDJbz8QAmvPVxmBhqzDUuycKc4jKVNMIaEgBESW2t7OLynldXvm8nZH5ozYpdYanYCl3yymI/fdQ4rr5xBxZZGHr9rC6/9rYyULBcrLps+5DYp2UY1MrKeWmdz35DHP391HhfdvJDTr5xhVo4L5qXR0+6j4UAnB7c3U3RWPqdfOZNLPllMa20P+8PrIUbGIjvcNmYtzSbgDfHKX0t5/ne7SEp3cdaH5nDOdXPxdPjYGp7pvuf1WlKz3ebfOS0ngexpyVRsbaKjqZfUqJ6SjCmJeDp8uBLtTF+cycylWfR1+Sl5s462Og/zTs9l/pn5uBLtA5bI8XT62Pd2PUVn5g279uiRJKY5mbk0m9K36zm82wiN0xdlkDcrBU+Hj7Z6Dx1NvUNe+wvmpVE/aPHtUEhn85MHeOh/NvH2PyvobO7l5QdL2PlyNbOX56B0xet/K+P53+8etj1+b5DNTx0kb1YKc1aO/WvM0ZAAd7JwZ4y6AgeQdN55eEtKCDQZbzRTk6eS4cpgR9OO42rGyitm0NnYy9P3bKe2rN0YWzLMYN7RmLU0m5VXzuDCdQuGHCc126icRVaTrz/YaQSp+QOD1PzVeaCMsnbMABf1QjknalzIBTfOH1BhjCUhxUH+bOOTe2HR6MJQLIXz06kt76DsnQZsdgszTssiNTuBq7+4lIs/sdB8cXMl2iksSqdyZwtKKT70zRWs+a8img5389hP3sPXN/wnw8O7W3Am2MgboaI4HIvVwpS5aVRsbaKnzTcg+GmaxqJzp9BS3cOODdU4E2xkFSSRPTWZUFCndn8HXk/A7GIHozvFnWyn6XCX2c06XAUOjACglDHTM6swCYtF46wPzqa71WtW4ToaerHaLGQWJOFK6p/80ny4G4fLSu7MFN57tpL97zaQWZA07ID54xHdjdpc3c2uV4zFXdOiApxSmGcm2P5SFW8+Xk5fj5+6ig5yZ6Ua3Yp5CYQCuvncDgV08wwklTuN6kn0m0NimgN3ioPmw91ml2lyhsvsso6EOq/HqAhGnvPJmS6627zmWVRyhpk0kZRmPP96O40A50qyDxkuUHxeAQFvyKzQaBbNvJ+cmckEAzotNT10NPWiFANuX3xeAR//6TmctmbqiB+YwAhwQV+I5sPdtNV5yMhPxGLRyJuVih5UvPP0QZIzXRTOT2fBWfkc3t3Cod0tbPz7fqYuSOcj31/NZbcWc+VnT8Od1F9Fm7Yok0/89Bzcyca29LyE8ELSPex6tQab3ULxeYXDNWsIV5Kd1e+bxdVfWmqOQTv3+nkxKzIWi0Z6foIx0Sqk09XSZ860jzZ/dR6r3tffdR/pCnzlr/vQdWWOIZu9PJvMgiR2vVJjhPNDXeZYxKlFGdhdVva/28j04kw+8NXlLLt4GkvWTmX+6jx2vFzFwR3N1Fd0sui8ggGvu3NW5NBY2UVvp5+0nP7XzUgYn7cqF6vNwvTiLCxWjbf/WRH+n8jB7rBSfH4BlbtazElJu16pQQ/pLL142qh/r7EsPr8AX2+Qtx6vMP7XZ6Wa3eN7N9aC6h//FlEwPx2/N0RzeAJVR2MvT/x0K1v/cxhnop0dG6p56H82s//dRs54/ywu/dQibvjeKk6/cgY1+9qHrZBv+89h+rr8nHPdvGPqkRlLEuBOFgkjn9B+sKQLzgfA88YbgPEmvCxnGTuadxxXM2Yuyeby/15Ma62Hzua+I45/OxJ3soPV75sVcxaouZRIuKusptSYzRXpVurfz03+HKMdgycwgPGmanNaze7To7X80uksvqDQXPD0WBSEV/IvfbuemUuyzMebOzNlyKDflVfMYOG5U7j+u6eTNzOVhedM4bJPFePzBKkf5vyNSlcc3tvKtEWZxzzrqmBeujn4euqCgSF53uo8bHYLLdU9TJmbhmbRzApG2WbjrB3R4800TSNnegpNh7vNcUJZBcMHuMiLcXQ3y9QFGSSmOoyKGkYXamqOG4tFIyWrv3u96XAX2dOTOeP9s+lpN7qfx/OTcaQb9bEfv8ehPcaZTSLPjYRUIxxETg6/5dlKdr5svFG01PSYy9VEuqcilayNj5bx56+/yYv372XvxloS05xmlx+Ef5/Tkmmq6qarxYs7xYHNYSU507jfSKiLhNpIBS45w4m3J0DdfqNyOtyA7sS0qArc/g7jbzzozSl/TippuQmUvFVHe4OHlCwXVrvxXCuYm47NbuH5/7fbrApFz+A8GpH/77ryDlrDAQ4gNzz2sbfTz/wz8tAsGovOmYICnvvtLpxuGxd9fBEZUxKZvTzHHN85HJvDSmpOArX72yl7p4F5q/NwJQ0/Rms4hUUZ3PC9VVz5udOYcdrw95kxJZG2eo+xTl9IDVuBHHCb/ERcSXY6GnuZtijDmFSA8Xw4bU0hrbU9lL5dj9cTMKuNVruFM66ZxcorZnD5fy8eMBv1zA/MxmK18MIf92C1W1gwaCJG9MSH6LHKOdOSQYOi8Bl5nG4bhUXpBHwhphalm6E4MpP2X7/Yxo6Xq9jzeg2zV+SY7T5WU+alkZ6XQG+Xn8IFGVitFjILk7DZLeZ4yKxBAS7yPKrdb/Qq/OMn79HZ3Melnyrmum+t5KM/OpMVl0/n4k8sZMVlxmx1TdNYeE4Bmgb7NtcPaYfXE2DHy9XMW507oLo7USTAnSwikxj00U1CcM6bhy0vj57XXje3LctZRnV3NS19x7c+zozTsrjy86eRnp84YMzaWBu8CG/NvjZyZ8Ve8mPl5TOYuzJnyAQGMF7s5q7MYenaqWaX1NGYsTiL8z4876hvFy0yDk7pirlRMxhjyZ+TxpqbigZUD/LnpGGxaOYssMEaD3fR1x1gRrj79NjamAYYVZvBv0en28ac042gGRkvl55njIk8EF44NHNQQMuebgy8rz/QSUq2e8gyFNHmrsrl6i8tNce3gVHtmrU8x5g04A3S0dhrvqmkhmcvh8JVn5zpKRTOTzeD55xRzMA7VnmzUpl/Rh7LLpnGx358Fmd/aI4ZdhJSwpWsLj+ttT34vSFWXjHDDKjTFhqPL1IdbG/w4O8Lsv/dRtLzEowZkOHxe4Mr0pHfZ1u9h5RwcEvKMO4vMhM1Emoj/ztJ6cZ+B3e0YLFoZBbGDlWJacZzreFgF91t3iEfksD4P1pwVj71FZ3UlnUMqHAmpjn5wNeWgzIqFJrGMb9pJ6Q4SM9LoHJnC54Onzl+LDHVaQ65KAoPOUjJcptdohffsijmuLWRZBYkUl/RSSigc9qa0VffBktMdR4xMGZOSTI/YACjCnCaplEQ/lssHjRrc+6qXJyJNt7+ZwUwsLv4tDVTWX31rCEf5hLTnKy8fDp6SDF3Zc6QwJqa7TY/BEcHuDkrc7nxB6sHVLkir/1zoj6AJqQ4uOG7p1MwP423Hq/A7w2x/JKhXcpHK3rWauTvbbVayJ6ejN8bwp1sN5/D5mNNdZKel8Ce12p55jc7ScpwccP3VplDK5IzXJxxzewBM8rB6D0oXJBB2TsNQ8Ye15V3EArqo1pP9USQZUROFgkZoHTwdvQv7DsCTdNIOu88up59FuX3ozkcLM1ZCsCOph1cNP2i42rO1KIMbvzB6uM6xpEkpDqw2S10NvdxaFcLzVXdrBx2yY9Mc2xULBd+dMF4NXNUElIcZBYk0tPuM1+AjobdaSVrWnLMCpxSisodLWgaI/4OjiRrqrEq/4zFWTG7BpZcOJWGA53MDFcZImMimw53k5DqGPJmkDvd6BatLmk7YtC3Wi0xx+vNWZ7N7ldrqNzZQldzH7PCS9akZLup2NZMc3U3ekiZlaU1H11gLrswXjSLZq7HNlhipALX6TcHxy84O5/VV8/C6wmYs+RcSXZcSXbaG3up2NZEMKCz5r+KyMhPpHJXy4AgGxGZyNB4qIu54Tchm91KQqrD7EKNVOAilblIZbBqbysZBYnDju1yuG3YHBZzbN/gha0j5p+Rx+anDuL1BMwqotm+6Slc9+3TeeGPe1BKHdfg7vy5aZSEx7ZlRI2tnLcqLzx+rP++165bSGdTL/lzYrd5JJkFSRzY1mxMKhqhQjwWIo+jcpcxJCA158gBDmDhOVPQrBrTBj0n7A4rC8+ewvYXq7A5LKOueC5ZO5W+rgCL18QOIQvOnkJX68EBAdNi0YYMSZi/Oo+gX2fe6QN7ENzJDq763BJ2vVqDrzcw4lqHR2Ph2fkEfEHmRt1f3qxU6is6yZ46dPIawJR56ezdWEthUTqXfXoxzhE+REYrOjOPl+431saMHjpTW9ZujGOdMfHVN5AAd/IwF/NtH1WAA6MbteOxx2h94EEyP34zCzIW4LA42N60/bgD3ImgaRop2W72bKxl54Zq0nITWHDW0LWXThbnXj+PgC80YG2xo5E/J5U9r9USCuhY7RZ8fUE2PVHB4b2t9LT5zJlvx8pi0bjhu6cPWynLKkzipjvOGLKt6XB3zO7RyIB5pYZ2b4xW3uw03CkOtr9UhR5eGR6MyovSFQfDyyVEqgbJGS5jTOQEcadEulB9NFf1kJTuNMepDf7bpOcl0NHQayyIm5dA7swUY1LOMO2PTGRAQXJUhTQl02VW4Lqa+3CnOMwqdSTIBQP6sOPfwPhfS0xz0tnUhzPBRsYwy68YlSZjBmJajMlLCSkOPvDV5cd9HsqCqAAXHawGL+0Suc+jrbxFRIL/sc6QPBqRgHV4dytWu2XUg/pH+nBafH4BO16qImd6yqiHTtjsVnNNx1gWnTuFBWfnH3Gsos1hHfb3plm0Mf+d2hxWVlw2Y8C2SGV7uNeX5ZdOIy3HzeILCo/qdXfWkmwcLiv7NjcMDHD7O8ibnXrMr+FjLT5aIY7MPJ3W6MfBJZ51FgkrV9L8q19Rcdll9D75DMWZi457IsOJlDsjBZvdwjnXGbPXjmcc2kQrmJ8+4hiZI5kyO41QUKcpvKDrzper2PtmHTnTUzj/xvlcdmvxcbcxMdVprng+GpEXzowYAS4x1WmOrRppAsNILBaN2UuzzXPw9nehGgGmYmsTrkS7GVQmmt1hxeGy0tvpp/5Ax4hVobTcBJoOdVFf0UnRmflHHBAdmcgAmF2oYCwn0t3ah7cnQG15h/m7AaMiGDls9hEWNI0Eikh3/XCKzy8AbfjxdMBxD+6O/N7sLitJ6cc2e3E0pi3K4Prvnn7E7s+xkJzhwu60mpNMjmfyV0RKpptzb5gXc+brsdI07YjhLV5MmZtGWm7CsENHUjLdLL1o2lEHLpvDOOf1ge3N5pIi3p4ArbU9A84UNNFOjr+SMNaBg6OayGBxuZj2178w9Y9/wJadTf13v8u6f7RxoGEv3qCXN7b+i7/9+84Bn5aVUtT1xM9JgM+/cT4fv/sclqw98uy1U11eeDZsfYUxNb707XqmLczg8k8vpvi8guOqvh2rSPdIVmHsgBapjA23hMhozF7e3/0aCXAp4e6d7jYvOdNjd59MlIRUJw0HO+nt9DNlzvCTfNJzEwkGdDSNUVUNIxMZgAGBNTm8Fty/frmNnjYfp185w7zOYrWYITrWBJ9okf1ijX+LNm2hMZtzuL/5WEjOcJGS5TLWFhzHv62maUNmL47bfVk0sxt1NOPfRmvxBYXHNXTiZOZKtHPTHWccU/f5kRSdmW+sfxk+/WJkeZ3hhhdMhMn9jngycYdT/1FU4CA8Fu7cc5nxyCNk33Ybee8c5Ef3+3jz0jPJuuk7LP7mX3lu52Pm/r/Z/hsu/eelbKzZOMJRTxyrzWLOdJvsElIcpOUmUH+gk6q9bfS0+1gUY7X4Eyl3ZgqX//fiYWd9zj09l+nFmUMGGB+NKXPTcCXZcSfbzZCamObEYgsvGhwn41EiElIc5kD1kd5YImvHTV2YaYanI4kE5ki3rPG9sRZcd6uXq76wZMibeXKmC4tVO+JZKZJGGeAAc9bheLro44s494bjmzwUbyLdqGMZ4MT4yJudSs70ZLa/eBg9pFO7P77Gv4EEuJNHpAv1KCpw0TSLhaxP30rm7+/FqjS8WoCmS5dj0+HlJ3+NJ+ChrK2MP+/5MxbNwu1v306Ht2Ps2i/GRP6cVOoPdLDnjVoSUhxMP21iP3lrmsaspdnDVkfnrszlqs8vOa4qisVqYcVl0weMf7RYtP7Tdo3RIOmxEhmP5UywjTiwPHt6MjaHhcUXjD6ELzxnCqdfNXNAAMifk0bO9GSu/vLSIWskgjEDesZpWUf8IDStOJMZp2WRfYzd3WMtf3bquJ3HcqKYFbjjXFZDjD9N01h+2XS6WrxUbGuKu/FvIJMYTh7OVNAsR12BGyz3vIvoePZf5CTkkGZJpPT1VUwt7+D3O3/Pew3vkepM5e7z7uYzL32GH7/zY352/s/G6AGIsZA/O43St+o5vLuVFZdNnzTdyksvGroQaEqWi47G3rh7kzdP8zYnbcRxTompTj51z/lHtbRNcoaLVVcNnImdkZ/Idd8+fdjbxBr4H0vh/PSYAVCMnchzNXpmrYhfs5Zkk56XwLv/rqSzqW/U/0snyuR49T8VWCxGN2pv63Efan7GfNJd6WgOB0nLV3BmQzIP7n2Qva17+faqb3NG/hl8Zuln+M+h//DswWfHoPFirORHjalacPaUCWzJxMudmUrGlMRxHeR+LCKL+eaPMP4t4ljWJRQnrylz07jhe6uYMg5jtsTY0ywayy6Zbq5FWhBnH3AkwJ1MEnOgp2lMD5mwejVptV3kBxK5YOoFXDrjUgA+UfwJlmYv5Qdv/4CtjVvN/du8bbR7j3yCYDE+UrPdJKY5mbowY9KPozn9yhnc8N3hK08TxRxLJm/SIobxnPwhxt68VbkkpTuxOSxHnAh0okkX6skktRC6asb0kImrV9EMPJjzNXIueL85VslmsfHrC3/NuufX8YUNX+CPl/yRt+ve5o+7/0iWO4snrn4Cly0+lm6YTDRN4/23LRvxrAaThaZpaNb4q2DNWp7DJTZLXJxqRwhxfKw2C2vXLaC7zRdX499AKnAnl9QC6BzbAOdatAhLQgLa9r3YLP2hQOk6fd+9k3vr1uC2ufnwsx/m3u33sjhrMdXd1fx5z5/NfUtaS9hUt2lM2yWGl5abcMwLl4rxZ3dYmbsyN66WNhFCHLvCooy4XERePsafTFILjTFwgT6wj033mWa34165gt533h2wvf2hh+l69llseXn87on/x+92/o7r5l3HWQVn8Y2N3+CRd//ERQ+X0XrjRXxh7x1YNAsbr3+dvsefJPHc83AUxse54oQQQohTkVTgTiYp4ZMZd9aO6WETV6/Gf/AggSZjfJ2vspKmX/4Sa1oawYYGpjXp/GrNrzir4CwAvr7y63zorRChf7/Ipl9+myR7En3BPnY8s56GO/6Xqls+QbB95HFyoY4OdJ9vTB+HEEIIMVlIgDuZpIYD3BiPg0tYZZyUvvOJf+EtLaX+299BczqZev+fAOh57fUB+6c09nDJliBBC1ywB/5xycO4bW48j/0TS0oKwfoGaj7/BXS/P+b9qVCIyg9+iLqvfW1MH4cQQggxWUiAO5mkhrslx3oc3MIFWNPTab7nHio/8EH6duwg73vfxb1oEa5Fi+h5fWCAa/r5L7C63HR+cx3uvhCON7ZyiWs5eTtqSLvhevJ/cid9W7dS/sXPEGwduuxJ3/btBOrq6H7pZTybN4/pYxHxq3frVoJtR7eO4fGeFF0IIU5VEuBOJimRADe2Xaia1crMJ/7JtAf+TMFv7mXa+vWkXHUVAEnnn0/fzp1ml6hn8zv0bNhA5qc/zdkf+yb26dNo//tjXLrbCgpaL11ByhVX8Mb7ZxF8/W3KL1xL409/Rqi727y/rhdeRHM4sE3Jp/End6FCIZRSdPzzn/S89daYPS7d56Nvz94xO96xUMEgve+9hwqFju84ShHq7Dy224ZCtD7wIMGWluNqw/HQ+/o4fPPHqVp3M6GenlHdpm/nTvavPJ2+vRP7NxRCiHgkAe5kYnMaa8F1Vo/5oe35+SSeeSYpF19M4upV5gy6pAvOB13H8+abBNvbqf/+97FPmULGuo+haRrp119P39atZD/zLjtma7weLGVD1QZ+s6CK2z5lpXplIW0PPkjD7XcAxuzW7hdfJPHcc8n92tfwlZXR9uCD1Hz2c9R/93vUf+9/ULputqvtb3+j5ktfpuuFF2N2yapAAM/mzTTceSd13/4OgUbjxMOhHg/Vt3ySQ9deS+sDD8Z8zKGengHBcqypYJC6b3yTwx/9GLVf+xpqmC7lIwn19FDzhS+w/4wzqfnil+jbvfuobt/73ns03X03Lb/9bczrA/X1dD777FFVu3q3bsXz7rtH3jHMV1YGgQC+8nJqv/RlVCBwxNs03XMPusdD1zOymLQQo6V0nerPfZ66731vopsixpkEuJNNaiF0jW0FbiSu4mKsmZl0v/QytV/4IsGGBqb8/OdYnMZipakf+ADY7aieHioumM2Lh1/k7vfuZm76XJatvJLvXdhE4s030fXcc3jL9hvVvMZGUi69hOTLL8e9bBlNP/s5PW++SfLFFxOsr6d3yxYA9N5emn91D90vvUTtl75E+bnn4YmaLRtsbqbikkupuvnjdDz6d7qee47K93+Aruefp+qWT9C7fbtx/Lvvpv3vjw14XH27dnHg0ss4fOONowoTR8sIb9+g67nnSLrgArqf/w/VX/gCvdu3U/ft71C28nS6X3n1iMfxVVRw6Lrr6Xn1NVKuugrP5s0cuu56qj79aXwHD46qLd0bXgGg48mnCHV1DbhO7+uj+tZPU/fVr+HZuHFUx+vbu5eqT9xijHP0eEZ1G29pKQBZn/0MnrfeouFHPx5xf8+779K7aTOay0X3hg3SlSqOSt+OHTT8+M7jrnyfjNoffZSeDRvoeurpUVe7xclJAtzJZhzWghuJZrGQdO65dL/4Ir1btpD/4x+TsHyZeb0tI4PUK67AMX06hRdfTUVHBQ2eBr63+nvcetqt9AX7eP4sJ5bERJrvvZfuF14Eu52kNWvQNI28239A4vnnMeNvDzPlp3djSUig69//BqDr+efRu7uZ9uc/M/VPf8KWnk7dN79JqKsLpRT13/8BobY2Cu75FfM2b2LmE//Elp1N7W1fwVdSSuGv72H6+gdJPP88Gm6/ncaf/Yyet96i85lnOfzRj4FS+MoraHvoYfPx9Lz+Og0/+jFt69fT/fLLdPzzCZrvvZe2v/1txN+TUoq2v/2NA1dexYErruTApZfR9dzz5Hz960z9/e/Iu+MOPBvf4PBHbqTrhRfQ3C4af/xjdK93yLGCLS00//a3VF5/AwffdzWhri6je/tnP2XOK6+Q/dWv0Ld1GwevvobGu+4eMYAqpeje8DKOmTNRvb10PPHEgOsb77wTX3k51qwsGu/+6RHDbLC9nZovfAGL243e1UXH44+PuH+Et6QEa2oqWV/4Ahm3fIKOv/+d3m3b+9sZDOLZ/A4qEEApRfO992LLySHnK7cRqKrCX1ExqvuJRff78dcM/NDjP3SIqls+ie/AgWM+7mSgQiF8R/m7D3V1EairG/19KEXjXUM/ZB2P5nvvpf2vf6XzyafG7JijFWxpGXaGvQqF8B2sHLf79ldX0/Szn+OYPh0VCNDz6mvjdl/HItjWZn5AF2NAKTWpvlasWKFOas9/S6kf5Sul6yfsLrteekmVzC9STff+Jub1us+nQj096kD7AVX8YLH6zhvfMa/74oYvqrP+dpaqvfceVTK/SO1bebqquvXT5vU13TXqgd0PKI/fo5RSqvYb31T7Vp6uQl6vOviha9WBq65Sevix9u7arUoWFauar31ddTz5pCqZX6Ra/vzAgLaE+vpU0//9n+rZtHnAtqrPfk6VLFioSuYXqZL5Rary+htUoKVFHb71VrVv+Qrlb2xUXS+/rEoWLlIlxYvN/aK/enftjvn4/XV16vDHP2Ec94YPq+ovfVlVfe5zqv0f/xj4e3zlFdX614dUsKtL9WzarErmF6nm3/1uwD7egwdV+ZoLVUnRAnXw+utV0333KX9D45D7DLS0qLrvfU+VzC9SjT//Rcx2KaVU3969qmR+kWp//J+q8sabVPlFFys9GFRKKdXx9L+N2//il6prwwZVMr9Itf71oWGPpQcC6tDNN6vSxaep3l27VeWNN6n9a9Yo3e8f9jYRBz/4IXXo5puVUkqFPB5Vtmq1qvrMZ83rm359ryqZX6TKL7pYNfz0p0ZbHnpI+RsbY/6eRqNvX5mqv+MOtW/ValWycJHylpeb19XdfrsqmV+kKi65VAU7O4/quLquq5Y//Um1rl+vvAcOmM/PaMGurpjbzWOEQuaxAq2tqnfHDtXxzDOq+Xe/V/V33KG8FRUxb9f9xpuq9jvfUYGmpqNq87HQdV3Vfve7qmR+kWp79O+jvt3hW29V+1atVoG2tlHt3/744+b/2OD/mdHoeOYZVfc/3zd/377Dh43jLSpW+889T4U8xmuLHgwqX3X1sMdpeeAB1fizn6lQX99R3X/I51O+w4dVx9NPq0MfW2c+j6NfLwJtbarlT39S5ReuVSXzi1TXyy8f9eM8Ej0UUodu+i+1b8VK5a+tVfvPPU9Vf/7zx3XMrldfVb5Dh8aohcp8HfbX1o7ZMUfiLS9XrQ8+qEI+3wm5v/ECbFEx8syEB6oT/XXSB7i3fqPUD1KU8rSe0Lv17t8/4htSxNu1b5thTCmldjbtVMUPFqsH3v2dKjvjTONF+p9PKKWUev7g8+qMh89QxQ8Wq+uevk41ehpV95tvmqEkVqBo+r//UyXzi1TpaUtU5UduNMPIaAS7u1X3G2+q9scfN1+kfZWVqrR4saq88SZVuvg0dfC661Wwu0cF2tpU765dyldVpQJtbWrfoMAR0b1xo9q3arUqXbZctT3yyKh+RxHVX/iiKl26TPnr65VSSvXu3qPKzjxLlZ151rBhcbC67/2PKplfpLrfeFPpuq7aHv27KjvnHNX9xptKKaWa7v2NKlmwUAVaW1Xn88+rkvlFquPfz6jGe+5RpYtPU5U33qT0QEDpuq4OrbtZla1arXxVVcpbXq769pWZjyfY3q4Of/zjZhhUSpmhr+Ppf8f4XfeY3+t+vyotXqwafvpTc1vTvb9RJfOLlLe8XPlralTpaUvUoY9+TB38wAdVyfwitf+CNeaL7sHrr1cHr73ObEfd7ber+h/9WLU98qjqfPFF1f7446p1/Xrlr6szj+/ZskWVLj5NlZ62RNXc9hVVuvg0VXf77UopI9DvW3m6Oviha1XJomJ1+NZbYz6Pgt09qvvNN1XLn+5XgZYWc3vv7j0Dgv2Bq943IFC1rv+LKplfpMrOOlsdvuWTquettwb9zYzgPeyHhQUL1cHrrzdDnlLGm3Pz736vSooWGAFhzYWqr6xsxOfG8Wp75FHjb3HOuapk4SLV/eabR7xN5ANDyfwiVX/HHeb21ocfVpU33qR6d+0asL+/rk7tW7FSHbrpv9ThT35KlSxYqDpfeGHIcUM9PcqzdZtqfegh1b1xY/92n89o3/wi1fHMM0opZbx2LFxkPt+b7rtPBTs71eFbPqlK5hep2u9+VwW7uwcc3/Puu/1/z/ddPSDsm4+ttHTAh5VQX5+qvOHDA/525RddrBp/+Su1f80aVVK8WDXceafxf7NwkSqZX6QOffRjav/5F6jKG2864u9yJP6aGtX1yisDtnX+54Xw/+fjSiml6v/3h6r0tCUq1NMT6xBH5Nm6VZUULVAVV16p9EAg5j6j+fAW0bun//+m8Ve/OqY2HY3ePXtU2arVqmR+kTp43fXKX1Mz7vc5XiTAnSoBbs+/jABXv+uIu8aLT7/4aaMKt/5+te/0VSrQ1qZ+tOlHqvjBYnXjszeqJ/Y/oU5/6HR10T8uUltq3zVfkEuXLlPBri7zOMFQUOl+vzp47XWqdMlS5T14cEza1/iLX5pvxMH29pj7NN13nyqZX6T69u5VShlvqE333adKihaoA1dfc0yfUn3V1ap08Wmq4pJLVflFF6uSogWqfM2FR/W4Qr29quLKK1XZWWer2m98w/i9FS9WZeecowKtrerANe9XlTcZbxa636/2n3+B+SJa89WvDQgmffv2DahSlswvUhWXXa6af/c7VX7JJaqkeLH55hD5HVRcfoU68P4PDAiu7f/6lypZVKza/m5UbfpKS83gGBFoa1OlS5aq2m9+S1V/8UuqdMlS5a+rU7quq+6Nb6i+0lJz3+b/9wdVMr9I+Q4dUpUfuVGVLCpWpcuWDwk+ZWedrTzbtilfZaUqW7VaVVxyqfn4ar/1bVW6bLkKdnWpjqefViXzi1TPps2q7W9/G1LF1HVdNfzkrgG/i7rv/Y95fd3tt6vS05aovtJS1frww6p06TJVecOHVcjnU56t21TJomJ16L8+qmq/9W21/9zz1P7zLzDfAANtbaq0eLE6dPPNqvFXv1KNP/+5al3/F9W14RXl3b9fhTwe4/c3v0i1P/Ev42/c16eqP/8F82/mee89tf+cc9W+5StUw90/VU2/vle1Pvyw0qOqDMH2dlV/xx3q8Mc/riquvFJVf+nLZiVqwPPH51P1P/yR6nj63wP+hp5t21RJ8WJ1+JOfUsHOTnXgfVerfStWKu/+/SM+H2tuu03tW75C1X7jm6pkwULVt6/MCPpFC1TJomJVsnCRavzlr5S/pkaFvF51+JOfUqVLlynf4cMq5PEYgah4saq/4w7lr6lRvspKVfvt7xi3Df8tShefZv6/tf/jH+bffv8Fa1Swq0uVnXW2qvrs55RS4Q9Jy5ariksvUyXFi1XNbbepkgUL1f4L1qjujW8Yv4PeXlV+8SWqfO1FqvOFF1TZmWep0iVLB1TJIs+T6FAaeV42/PSnqv2JfynPtm1m6A62txvVpnCVt/EXvzQDd8sDD4Qr+sO/hvdsfkdVf/nLqvWhhwZ8uPBV16i6//m++fvo3bHDvK76C19UZeecY+7fs/kdVTK/SHU+//yIf7NYQl6vqrj8CrVv+QqzGj6kjZs2qX3Llo/6+FWf+azad/oqdehj61TZWWcPeL5GBNraVMjrNX/WQyFV85WvquovfHFI6B5J765dat/pq1T5mgtV29/+pvatWKnKVq0eEnpjaX3oIVV3++3Kd/jwqO9vvEmAO1UCXPUWI8Dte26iWzJqpa2lqvjBYnXP1nuUHgiof5T9QxU/WKzueucu5Q8Zn+BKWkrUhX+/UBU/WKx+8wnjRePQt75uHuPd+nfVmQ+fqZ7Y/4QKdnePaVk/1NurWv74xxG7pYKdnWrfytNV9ee/oHzVNWZXSc3Xv65Cvb3HfN+tDz6oDlx9jar+8peN7tLGod2lR9JXVqZKT1tiVBt+/WvVt3evWVUc3M3c8eST6vAnblGebdtiHqvrpZdU61/+qjr+/Yxq+/vfVeWHP9IfjrZuHbJ/22OPmb+HvpIS1fKn+8032oMf+KBSSqn2fz5hVNsOHBhw2/of/dgMSU333Tfs4/NWVPRXgooWqM7nn1d6KKT8tbWqb+9e5a+pUX0lJar84ktU6eLT1P7zL1BlZ5w54DkS+fTf+uCD6tDH1qnytRcpPRRSuq6ruu//wOgmfORRpZTxAh55TN1vvKlqv/0dVVq8WPkbG1Wot1ftW3m6qvl6/3MzUump+cpX1f7zL1DlF11sdst2vvii0WUWfuOIHDs6oA6mh0Kq8vobVNnZ5yh/Q4PxXCtaoFoeeMAMWf66OlV5/Q2qNKqCV/P1rytd15Xu86lDH/2YKilerCqvv0FVffq/VcmCherQTf81oBqj67qq+drXzdtXffq/Vc/bb6u6//m+Kl223HgcHR3G/dXWqrKzzzEqg1FBr+PJJ1Xzb3+rdJ9P+Q4dUiULFqrGn/9cBdvbVdmq1ergBz6oSpctVwc/dK3yNzYaYWxQ8G79y1/N4wU7OoyAUrzYCCkLFqrS05ao+jv+V3VteEX17tmj9i1foQ7f8kmlB4Oq4tLL1MEPfFD1vGOElcjztfu115RSymhT8WJVduZZyvPee8ZzYft2VXHZ5cbv7LbbVO13jDb1bH5HKaVUoKlJHbzuelWycJHq+PczqvOFF1RJ0QJVtvoMVVK0QPXu2GFU5Veerqo+/d/D/x11XQXa2oZU5YPd3Wrf8hWq5itfHXIbb3m5WSmMfEg5eO11qu2RR9XhT9yiSooWqNLixaruBz9QpUuWqrof/CB8zB7j9/S/P+y//2BQlZ15lqr+8peHbeNwmn79a+P3uPENI3CtWm0+FyKPrfL6G8wP2n0lJSMeL1K1brrvPtW9ceOAiqlSxnO+df16VbpkqTp47XXmh41INdv4gH2V8lUPrKIFmppU9ee/MCDI+mtrjfB20cVm1c136JA68P4PGB/Gbr992Ndsf0ND///UwkWq9hvfHFBEmChxE+CAqcCrQAmwF/hSeHsG8BJQHr5MD2/XgHuBCmAXsDzqWOvC+5cD60Zz/yd9gOuqNwLcO3+Y6JYclW+8/g218q8r1du1b6uVf12pPvnCJ1VIDw3Yp8vXpZ6qeEp9/+Fb1GurF6jP/+F9qsPboaq6qtQ5j5yjih8sVqc/dLo63Dkxn4wi47RKly1X+5avUG2PPXZUXabjyfPuuwPG/bX8+QHzhe94P0n6qqqGHSem+3yq4c47VenSZeb9VX/py6rlj380Qtv+/ar+hz9SpcuWD+mm9NfUqJKFi1T5hWtHHHek67pRQRn0Zj9YoK1NHbrpv1TpaUuUZ+vQgFr5kRvN6m7zb3/bf/xAQB2+9VYjfPzqV6pk4SJV9d+fMaspvsOHjet+9jNz7GXkzT6i8Z57zOpQpEqrlFH1LDvnHPON/uAHP6QOvP8Dwz6GiN7de1RJ0QKjArJwkep4+ukRfz/Nv/udOZ4x0q3e8dRT5j6dzz6rShYuUpUfudHsGo8MR2i67z71/9u77/goqrWB47+zqRBKIBiaUgUEFghNRBRBxYKKehUQG1aUKyqiWK5Xfa0XFHtvGKTYC0WQDtKlSknoEAKEhFTSs+V5/5jNkgpBE5Ykz/eT/ezu1Ofs7GaeOefMTFJkpER3ibDK0CVCDj39jOQeOFBoPfm1UPnfs7wjR7wHDntvulkOPPSQRHfq7D0IyU9Wd/a9pNCBSdZff0nKjz/K0Y8/sWoOXYX/D4hYCeqR11+X+DffKnZglb9TPzh2bKEapvxayp39+hf6rmVt3lKsH6krJ0cSPvhAojt19tSsvVRovDM9Q/bffoc3Ydo3ZKg4jh6VnX0vkT2Drpe4l16WqPYdTlojWZoj/xsnUR3t3mZ/t9stKT/8INFdImTH+b0k8cuJ4srOltSZs2THhX28zeYJ777nnefg2LFWf+HsbG9/1sx16wqt5/DzL0h0126SvW2b5OzeXWoLgyszUw7/9zk58NBD3hrPQ08+KSKeGvT2HSTu1Ve906f/scz67nzwgey8pJ/s7N+/UG1+QW63Ww488KBsP7+X1TfU5ZJdlw/wtgxkb98u+2+73UrAh90qUe07SMyIEZIdFSXRnTrLgQdHSsaKFbK9R0/ZcWEfydq40Vpubq41fbvzZGf//uJMSxO32y0xd98j0V27Ffu/58rNlSPjxntrRZOmTCmUlIqIxL36qkR16CiZ6zdY26hDR2/Xi3xZW7b+o4P2v+NMSuAa5ydhQG1gJ9ABeB142jP8aWC85/VAYI4nkbsAWCPHE769nud6ntf1Trb+Sp/AuVwiL4aJzHve15GckgPHDkjE1xHSeVJn6fttX0nIPHEn7KWxS6Xr111lyMwhcsOvN8iF0y6UP+P+lAunXSi3zrpVHK6S+2RUJGdKiuzoc5HE3H33Gd+fwu1yScy998newUNOy/qcqamS+MWXcvTjj8XtdIojMdFqLpswQfbdepvsu2VYifOlzZlTpv5++X0XT8btdJbaeT7tt9+sJPO89oX6y4lYfaz23nSz90i/aHNNftPg3psHy64rriiWuLtdLol/8y1Jmzev2Hrj33pbotp3kPSlS60kdNKkk5ZDRKxaFnsnOTZ//kmndbvdVs1Vfh+jt94uNk3anN+9TW87el9o9Qd78qnjnf8PHJDUmbNKTdZdOTme7/89VnzPvyBR9k6SNHmKdaJIu/Pk8PMvHI/J4ZCE9z/420lOqWV1OLy1KbuuuMKbrOUeOCDREV3l6KdlP7jN3bdPjn78ScnNy1lZcmDkv2XPoOu936n8E7qi2p0nh/7zn2LzlHm9sQetROXe+yT+7bcldtQoq9XhrruKJazOY8cka+vWYoluxsqV3pqsA/9+SHb2vaTUaQo+9lw3SOJefkVy9lhdNdy5uVbtXvsOsue6QbLz4r6y58YbC/2O8pttU6dPt2rfht5iJcq5uZK1ZatEd+4i23v0lH1Dhsqhp56WtNmzxZWZKbkxMd4TvI5+8ql3efk19fnjtvfoKSk//mT14/X0vYzu3EV2XNjHmxjm7N4tuy4fINGdOlsnrnhORIp/8y2J6miX2NGjJfnb76za9GnTSv3sM1as8H5/ojt3kfi33ha3yyWOhASJ7txFDj1zfLvGvfyK1RXAU2OefwC39+bBpSasFeGMSeCKBQDTgQHADqCxHE/ydnhefwoMKzD9Ds/4YcCnBYYXmq60R6VP4ERE3u4k8uO9vo7ilI1bM07skXZZGru0TNPnJ3FdJnWRVYdXiYh14oM90i6PLX5Mxi4dK7f/drvM2198p1lRSuvMeyZyOxwl9jM5XQ6MeEB29r1EtnftVqhpx1fceXmys1//QmdBF+Q4elTiXn6lxDMVC3bOL7gjKovc2FgraTq/l0TZO5X57Ey30ymOpLKfrOR2OOTQk0/J4f/+t8RaLRGRvLg4SfnxJzn4xFg59OSTp3x23tHPPvMmDVEdOkrcy69Yy/XUmJ2Os2NFRLI2bZLoLhGS8ssvhYY7U1JO6cSmsiiarB946CGJ7trNe/LR35VfUxrVvoNER3SVhPc/OKXY3S6X7OzfX/bdMsw6Sei110qMPWPlSkmb87ukzpwlRz/+2Kqh6hIhUfZOEv/mWxI7erTV5/IEZwA7U1OtGsl253n79iV/8413fMaaNXL4+Res5lbPwUF0RFcrsevWXZImF+7L50hOtloyep4vCe+9X6wmLP6ddySqfQc5tnhxoeGO5GRv15D8/ociIkc/+dQa1tEu+4ffVer3v6CsrVvl4ONPWK0Go0dL3IsvSVT7DoW6XjhTU2VHrwtk/223S9bmLRLdqbPsuf4Gie4SIbsGXFGuXXlOpLQEzljjfMMY0wL4A7ADB0Qk1DPcACkiEmqMmQWME5HlnnELgaeAfkCwiLziGf4ckC0iE060zh49esi6yn4dmq+uAXHBPb/7OpJT4nQ7OZB+gFZ1W5V5nr+O/kWmI5MLm1zoHfbs8meZsWcGTUKaYDM2jmQe4f3L3ueiphdVRNjqbzo2Zw6HHhsDQONXXyH0ppt8HBE4EhKwBQfjV6fOKc974N77yFy1inMXLyagYfipz7tiBbUHXM7Z779/yus+U7gyMtjd/1LcmZmY4GDOnTcX/wYNfBKLOysLW82ap3294nDgTE4moGHD077uoo6+9x6JH30MQItvv6FGRESZ5nMmJpIw4U3Sfv0VgPCxTxB2770nnEccDuL/9z9Spn2Df6NGtJ43F1tgYPHpXC6y1q3n2OzZIEKDhx4q8ffiOHQIW926+NWqVeL6XMeOlfg7deflEf/aa7gzs2jyv9cw/v6Iy8WBu+8hZ9s2Wk6fTuDZTcvwKXiu3znxKxLeeAOAutcPosn48YWmSfn2O4783/9hCwnBVrcOLX/8EceBA8Q+OBJsNlr9Ngv/evXKtL6/yxizXkR6FBvuqwTOGFMLWAq8KiI/G2NS8xM4z/gUEalXHgmcMWYEMAKgWbNm3WNiYiq2cBXt5xEQswoeO7VbKlUVbnGT68qlhn8NMvIyuGfuPexL28f4vuPJcmYRnRRN23ptGdhyIAF+Ab4Ot9py5+Sw66KLcWdk0PLnnwju0MHXIf0jjrg4cvfspdZFfU553vSFCzn40CjO/uRjavfrV/7BnUYJb79D0qefEjbyQcIffdTX4VRreQcOsOeKK/Fv0phzFy703gKxrLI2bMRx6BB1r7u2zPOkL1iAf3g4NTp3PtVwK5Q7Lw9XaioB4ad2cAXWwWbi559z9ltvEdiiRaFx4nKx7+bB5O3dS/NpU6nRsSMAufv2kbliJfVvv608wj+hMyqBM8YEALOAuSLylmfYDqCfiMQZYxoDS0SknTHmU8/rbwpOl/8QkQc8wwtNV5oqUQO38CVY/g48dxRsfr6OxueSspMY/vtwYo5Zibm/8ccpThrWbMj9ne5nSLsh3n9s6XnpfL/je25scyP1g+v7MuxqIe75F0ibNYt2q1dhSjhary5EhNxduwhu29bXofxjrmPHSI6cRP177sGvVoivw6n24v/3PwJbt6bekCG+DqXKcqak4EpJIahV2VuPytMZk8B5mkcnAckiMrrA8DeAJBEZZ4x5GqgvIk8aY64BRmGdzNALeE9EzjfG1AfWA908i9gAdBeR5BOtv0okcGu/hN/GwJhoqNPE19GcERKzE1kdt5o2oW1oHdqaVYdX8fmWz9mYsJGPL//Y27z65ro3idwWSXiNcMb3HU+PRsV+E+S6ctmdupvY9FgubHIhdQJPvblNWVwZmTiPxBF07rm+DkUppSqlMymBuwhYBmwB3J7B/wHWAN8DzYAYYIiIJHsSvg+Aq4As4G4RWedZ1j2eecFqiv3qZOuvEgnc7oUw5V8wfBa0vNjX0ZyxHC4H1/5yLWE1wpg6cCqpualc+dOVdDmrC3GZccSmx/JEjye4o8Md3nneXPcmU6Km4BQnAM1qN+P9S9+nVahvjryUUkpVb6UlcP6nOxBPX7bSGuovK2F6AR4qZVkTgYnlF10lEe7pS5QQrQncCQT4BXB/5/t5cdWLLDu0jE0Jm8hx5vDM+c/QMKQhzy5/ltfXvs5ZNc7iqpZX8cuuX4jcFsnVLa7m8uaXE+gXyAsrX+C22bcxvu94+p7d19dFUkoppQAfnsTgK1WiBk4ExreAjjfAde/6OpozmsPt4LpfrqNWQC0OZhzkoqYXMeES6zyXPFce98+7n62JW3nq/Kd4fe3rRIRH8Mnln+Bvs45t4jLieHTxo+xK3cWUgVPoGNbRl8VRSilVzZRWA2fzRTDqHzIGGna0auDUCQXYAhjReQQ7UnaQ6chkROcR3nGBfoG80/8dwmuG8/LqlwkNCuX1vq97kzeAxrUa89mAzwgLDmPs0rFk5GX4ohhKKaVUIZrAVVbhHawErprVoP4d17W+jhZ1WnBliytpW6/wWYD1guvx4WUf0qtxL97u93aJZ6aGBluJ3eGMw7y06iUK1lpnOjJ5YukTfLjpQ1xuV4WXRSmllAIf9IFT5SS8PeQeg7RYCG3m62jOaAG2AL6/7vtCNWsFtQptxRdXfHHCZXRr2I2HIh7ivY3vUb9GfUZFjMKNm5HzR7IlcQuCsDFhI6/3fV0vT6KUUqrCaQJXWTX09MVKiNYErgxq+Nf4x8u4t9O9HMk8wtToqczdP5fQoFD2H9vP2/3f5ljuMV5Z/QpDZw3l+2u/p15w4Stz57nyiNwWSZBfEMM7Dv/HsSillKretAm1sgpvbz3Hb/NtHNWIzdh4rvdzfHPNNzSp1YTY9Fje6/8elzW7jBvb3MiXV35JfGY8k6MmF5pvQ/wGbp55M+9vfJ8J6ybw484ffVQCpZRSVYXWwFVWwXWh7jmQEOXrSKodewM7U66eQpYzi5CA41eijwiP4IoWVzBt+zSGdxxO3aC6LIxZyGNLHqNxSGM+vOxDpm2fxqurX6V5neb0bNTTO2+WI4ttSdvoENah0DJPJD4znnc2vENidiJZziyubH4ld3a8s9zLq5RS6syjCVxlFt5ez0T1EWNMiYnWiM4jmLt/LlOipzCo9SCeW/EcHcM68uWVX1IzoCZdw7ty2+zbGLNkDFe2uJKaATU5lH6IZYeWke3M5tzQc/nk8k9oGHLyG2VPWDeBRQcW0SGsA1mOLCasm0DnszoTER5R4vQiwo6UHeS6culyVpe/Ve48Vx6BftX3llhKKXWm0CbUyiy8AxzdAS6HryNRHm3rteXyZpczNWoqTyx9AoA3LnmDmgE1AagdWJsPLv2AJrWaMG//PKZGTWVd/Dor2bvgOQ5nHOb2ObezKWET646s47e9v5GUnVRsPdsSt/H7/t+5y34XkwdOZsrAKYTXDOfFVS/icBf+PqTmpPLO+ncY+PNABs8czJ1z7uT3fb8XW2ZUUhTD5wzn+x3fl1i2xQcWc8G0C9iWWPZm+2xnNvvS9pV5eqWUUmWjNXCVWcOO4HZA0u7jfeKUzz3Q5QEWHFhAVFIU7/R7h7Nrn11ofLM6zfju2u+870UE645x0KlBJ0YuGMkdc47f3qtV3VZMHjjZe09WEeHt9W9TL6ged3e8G4CQgBCe7fUsjyx+hEnbJnFfp/twuV38tOsn3tv4Hhl5GfRu0pv7Ot3HjD0zeGbZMwT7B9PvnH6k5abxddTXfLnlS1ziYnfqbga2HEitwFreGDLyMnhlzSs43A6+3fEtLzd4+aSfQ0ZeBiPmj2BL4hb6NOnDiM4j6Naw20nnU0opdXJ6J4bK7MgW+OQiuHki2G/ydTSqgHc3vEutgFrc2+neU573SOYRVh1eRcOaDclyZjH2j7H0bNiTjy7/CH+bPysOreDBBQ/y9PlPc1v72wrN+9jix1h6cCmNQhoRnxlPnjuPHg178J9e/6FNvTaAlVjdN+8+dqXsIrxmOAczDgIwqPUgrm11LSPmj+DRbo9yX6f7vMt9bc1rfLv9WyLCI4hOimbRkEXUDqxdahmyHFk8MP8BtiZuZXC7wczdP5fknGRuPPdGnuv9HAG2gFP+XJRSqjo6Y25m72tVKoFz5sKrjeGix+Cy53wdjaogv+z6hedXPk+/c/oRYAtgddxq6gTWYeYNMwnwK5wIJWQl8NKql6jhX4NGIY2IOCuCS5td6q3hy5eWm8bLq1/GLW46hnWkR6Me3n5xIxeMtJpob/qdmgE12Xx0M7fPvp1h5w1jUOtB3PLbLTzb61luOe8WwOoXdyTzCIczD3M4w3osO7SMHck7eOOSNxjQfADZzmw+3/w5n2/5nD5N+/DWJW95m5VFhD8O/sEfB//g6pZX06NRsf9TFSoxO5H4rHha121NsH/waV23UkqdjCZwHlUqgQP4sBfUawG3fnfSSVXl9fb6t5m4dSJNazWla3hX7uhwBx3COlTIujYlbOKOOXcwuttoQoNC+XDThxhjmH79dEICQhg6ayhucfPDdT+w6MAinln+DNnObO/8NmOjUc1GPNbjMa5qcVWhZf+08ydeXv0yzes0p0/TPjSv3ZzZ+2azIWEDfsYPl7jo3rA7Xc7qwv60/aTlpTG62+gST8zIdeWSmJ1Ik5AmxRLUspq5ZyavrH6FLGcWNmOjRZ0W9Grci75n96VNaBuynFnkufJoHdq60IWgU3JSCPIL8iahSilVUTSB86hyCdz0URA9A57cBzY/X0ejKoiIkO5I9/aDq2j3z7uf1XGrAejcoDPPXvCsN2H8fsf3vLz6ZW5vfzvTtk/DHmZncLvBNK3VlMYhjWkY0vCETaR/HPyDjzd9zO7U3eS4cggLDmNkl5Fc0+oapu+ZzsQtE0nOTaZZ7WZkODI4lnuMd/q/Q5+mfbzLWB+/nv8u/y8HMw5SO7A29jA7j/d4nHb125W6XhHhmeXPEJMWQ7v67chwZDB3/1y6hXfjlvNuYU/qHrYlbWPdkXXkuHIKzWsPs/NSn5doVqcZn/71KV9t/Qo3bs4NPZcLGl/AqK6jyuVi0eUlNj2WX3b9wu0dbi/znUE2JWxiffx67up4F36V/H/Jmrg1rDi0gtHdR2Mzeq6eqtw0gfOocgnclh/hp3vh/kXQtLuvo1FVxPbk7by74V0Gtx1M/3P6F6rhysjL4NIfLiXbmV2sOfRUuNwu4jLjCKsRVij5cYsbt7jxt/mTlJ3EgwseZHfqbu7ueDchASHEpsfy866faVqrKbe2v5W9aXtZdGARdYPq8sN1PxDkF4Rb3EzcOpGu4V3p3tD6XczbP4/Hlz5Ou3rtOJJ1hIy8DO7vfD8PdH6gUO1ajjOHtUfWcjjjMCGBIWTmZfLhpg9Jd6TTsGZDDmUc4tpW13J27bPZfHQzqw6vovNZnfng0g8IDQ7F5XaRmptKWI2wf7AFjjuSeYTZ+2ZzQeMLCtW6ro9fT3iNcM6pc453mNPtZGr0VD7Y+AE5rhyubHElEy6ZAFj37f1086cMazeMxrUaF5rns82f8enmT3GLm5FdRvLviH97x21K2MTR7KMk5yTTvWF3zqt/XrmUq6JsTNjIiHkjyHHl8F7/9+jfrL+vQ6pQWY4sAmwBxbpTVITknGS+3vY1beu1ZWCrgac0r4gwcetEVh1exTO9nqF1aGtEhFl7Z/HnkT957oLnKvQSRZsSNmGMwR5mL9MBioiwLn7dKV2bs6JoAudR5RK4jASY0AYuewEuHuPraFQ18fW2r4nLjGNMjzEVfkLCsbxjjF48mrVH1gJWE+3gtoMZ032MN3FceWglDyx4gBGdR/Bw14e9Tc61Amox9ZqpNAlpwvW/Xk+twFp8f+332IyNPHceQX5BZYohOSeZcX+OY3vydp7q+VSh2sD5MfN5+o+naVKrCT0a9WDRgUUk5yQzoPkAHuv2WKEEC2D5oeXEZ8ZzY5sbS60dcrldrDy8kh92/sDSg0txi5ua/jX56PKP6N6wO1OjpzLuz3G0qNOCnwf9TIBfACLCw4seZunBpfQ7ux/n1DmHyVGTea//e/Q9uy+jF49mycElDG47mOd7P+9dz4j5I/jzyJ9c1+o63LiZvXc2H172Ie3qt2Ps0rFsSNjgjatRSCNm3jDzlPsK7kvbx8aEjfyrzb9OOF1UUhTj/xxPszrN6BDWgV6NetEqtFWZ17MjeQd3z72bsOAwcl25NA5pzKSrJ51SrOUlKTuJWXtnccO5N1A3qG6hcW5x8+vuXwn0C+TaVtd6h6fnpZOck0zzOs3LtI7UnFSGzBoCwJM9n+SyZpf97e4EJ5LryuWrrV8RuS2STEcmAbYAfrjuB1qHtgaspDnTkclFTS8qcX6H28FLq16yymwLxGZsjO4+mnVH1rHgwAIAXu/7Ole3vLrcYwfYmriV22ffjktc1AmswyVnX8KYHmNoUKNBqfMsiV3Cw4seJjQolLvtd9O+fntm7Z3F4tjFPNz1YYadN6xCYi2JJnAeVS6BA/joQghpAMNn+DoSpSqEiJDtzMZmbPjZ/EpMGp9d/iyz987mtva3MSlqEgNbDvSe8NHvnH5Ebotk4pUTC90Bo7ysj1/PI4sewel20vfsvjQOacy3O77F4XYwutto7/1v4zPjGfTrILKcWfRs1JNX+rxCk1pNCpVzctRkpm2fxqGMQ9QPrs+/2vyLfuf047kVz3Ek8wgDmg9gxp4ZdAzryLakbTzR4wmGdxzuPdnl8e6PM7zjcJxuJ0N/G0pabhr9z+nPdzu+o2mtphzLPcaiIYsI9g9m4YGFjF482ntGc7Yzmztm30FcZhz+Nn+yndmM7TmWbuHdOJRxiIcWPsSoiFE80OUBwEqu03LSvEnqwfSDfLDpA/JcebzR9w38bH443U6GzBrCrpRdhWrEErMTWXl4Jde0vAY/mx+5rlyGzBzC0eyjBNgCSM5JBqB9/fZc1/o6hrYbWmINTaYjk6WxS1l2aBlLYpdQM6Amk6+ezOLYxYz7cxyTr55crA+lw+3AYArVvJbVb3t/Y0HMAp7v/Xyxex7nc7qd3D/vftbFryO8ZjgvXfiSN+nfn7afF1a+wIaEDQTYAvjtxt9oXKsxIsKI+SNYe2QtY3uO5dbzbj1hMiYiPLbEOuu8RZ0W7E7dTZ8mfRjfd7w3YXSLmxWHVtCufjvCa4YXW8Yvu35h7v65jIwYWerFvdNy03hk0SNsSNjAZc0u4/b2tzNmyRia1GrClIFTWHl4JY8uftT6vrUbytieYwsdGCVmJ/L0sqdZE7eGB7s8yJC2Q3h2+bOsiltFgC2AUV1H8f2O72kc0pivrvoKsK5z+fra1xnTY0yZLzqe48wh05FZrOY725nNkJlDyHZm82i3R1kTt4a5++dSJ6gOb17yZqkXPr9n7j3EHIuhTb02rDi0AoBaAbVoFNKIvWl7+fCyD0tNWMtbaQkcIlKtHt27d5cqZ87TIi+Hi+Rl+zoSpXwmJTtF+n7bV+yRdnlw/oPicDnkz7g/JWJShNgj7fLY4scqdP25zlzJceZ43ydkJsgjCx8Re6RdlsYuFRGRxxY/Jt0nd5cvNn8hvab2kl5Te8nyg8u980zcMlHskXYZPme4zNk3R/Kced5xiVmJcuP0G8UeaZfnlj8nDpdDHpz/oFww9QKJToqW3lN7y/A5w8Xldnnn2ZywWTpFdhJ7pF1eWfWKrDm8RuyRdvltz28iInLXnLtkwA8DxOFyeOeJSYuR3tN6y6BfBsnulN2Fyjh60WjpOaWnxGfGy9ajW72f97U/XytPLn1Sun7dVbp+3VXskXb5YvMXIiLy3fbvxB5plwunXShX/HCFZDmyJDMvU26ecbPYI+3y1B9PicPlkAlrJ4g90i4rDq4Qt9sth9IPyZSoKXLLzFvEHmmXe+feKxl5Gd5Y3G63zNk7R/p/11/skXa5+JuL5ek/npZ9qftERCQzL1P6fNNHHl30aKEyLIpZJH2/7SujFo4St9td4rZMy00rcdyelD3SfXJ3b5kPpR8qcf53178r9ki7vL/hfbn+l+vFHmmXgT8NlEu+vUS6TOoivaf1lsitkRLxdYS8uPJFERFZdnCZdzp7pF3+s+w/ku04/j89Iy9Dxi4ZK6//+bocyTgiP+z4QeyRdoncGikOl0OmRE2Rrl93leFzhkuuM1dcbpc8v+J5sUfaxR5pl1tn3Sq/7vq1UFm6fd1NOk/qLPZIuzz9x9OSkp1SqByH0w/LoF8GSdevu8qcvXO8w+fsmyP2SLs8seQJ6fZ1Nxk8Y7CM/3O82CPtMnjGYJm5Z6YkZyfL0til0vfbvtLt626F1u1yu2TWnlmyK3mXiIh8vvlzsUfaZU/qHnG5XTJk5hCxR9qlx+QesiBmQYmfsYhItiNbopOiZdyacdJ7Wm+rnL/dKpO2TpI9KXvE7XbLK6teEXukXVYfXu2db3vSdrnqx6sk4usIGb1otDyx5Al5edXLkpydLCIiUYlRYo+0y1dbvhIR63c0f/98yXZkS2Zeptw0/Sa5YOoFxX4fFQVYJyXkM1oDVxXs+B2+GQp3zoBWl/g6GqV8ZuWhlczcO5P/XvBfb7+VH3b+wJdbvuTLK7+kaa2mpzWeHGcOd865k4MZB3mk6yO8uuZVHu76MCM6j+BQxiEeXfQo+9L28d6l72GMYeSCkVze7HImXDKhxNqXtNw0NsRvoN85/TDGsC9tH/+a/i8C/AJwi5ufBv1UrPntq61fsTdtLy/0fgGbsTHw54E0q92MMT3GMHjmYB7v/jh32e8qNE9qTio1A2oWq/GKPRbL9dOvp1ODTkQnR1M/uD7DzhvG8kPL+evoXwxoPoCHuz7M62tfZ0nsEr644gtGLx5Nq9BWPBTxEPfMvYf7O93P/mP7WRCzgGtaXcOsvbPo0bAH6+PXc3Pbm73NuwXN2DOD51c8T7v67Xiq51NEJ0ez8MBC1h5ZS/v67b21hEX7Nr2/8X0+3/w57/R/Bz/jx4IDC/h196+EBYeRlJNUqNlub9pepu+ezpLYJexN20vDmg3p07QPlzW7jIubXozT7eS22bcRlxnHcxc8x/+t/D9qBNTg/UvfL9Q3ccWhFYxcMJIb29zIixe+SK4rly+2fMHe1L3UCapDWHAYQ9sN5ayaZ/HK6lf4aedPzLhhBo8ueZRsRza/3vArE7dO5KNNH9EhrAPv9HuHukF1GblgJH8d/QuwbuVnw0b3ht35ZMAn3qb42Xtn89Syp7iqxVXUDarLdzu+484Od1I3qC5z989lZ8pORkWM4t5O93LH7Ds4mHGQaQOn8cvuX/hq21dc2eJKxl08DrCaTW/49QbSctN499J3i9Vcj1kyhvkx82lXrx1fXPEFocGhLI1dyourXuRo9lEMBkFoW68t4y8ez7n1zi3lV2LV0g34YQDD2g+jTWgbnl/5PE/2fJI5++awNXEr1597Pc1qN6NuUF12p+5mW9I29qft51jeMQD8bf4MaDaAVqGtWHhgIduTtwMQGhRKam4qd3S4gyd7PllonWm5aby25jW2J29HEGLTY+lyVhc+H/A5L6x8gYUHFjJ/8PwSTxyLy4hj2G/DqOFfg2+v/bZYE3l50yZUjyqZwOWmw7jm0OdRuPwFX0ej1BlHCtzt4nSLTY9l6KyhpOel07JuS3687kdvYpSak8p98+5jX9o+gv2DCa8ZztSBU0/ppJAJaycwKWoSY7qP4W773Sed/qNNH/HJX5/Qu0lvNiZsZP7N809pB/TW+rf4autXnFf/PD667CPOqnlWsWmSc5K5cfqNHMs7hsvt4rtrv6N9WHueWfYMs/bOAvDGOzlqMq+vfZ2mtZry06CfSu0w/sfBP3h8yePeM4QbhTTiro53cUu7W0rtlJ6UncSVP11JrisXsPpP3mu/lxGdR3DX73cRlxnHjBtmsPnoZh5f+jgOl4PujbrTo2EPdqbsZPXh1aQ70mlTrw0t67RkXsw83un3Dpc1v4ydKTsZuWAkydnJjOgygiFthxC5LZJp0dNoXrc50wZOO2lfwfjMeAb+PNB7Qe03LnnDe+mdxQcW88zyZwjyC+LsWmezLWkb4/uOp2NYRyK3RbItcRvvXfpesc9/4taJvL3+bQDu7ng3j3V/DGMMLreL51Y8x8y9M+nUoBNbErcUWt9b699i0rZJ/HL9L7Sq24pJ2yYxYd0EPhvwGb2b9C4We2pOKt9s/4ah5w0tdKazW9xEJUWx7OAyAvwCuKPDHWXqa/r4ksdZHbeaQL9AmtZqyuSrJ5PjyuGlVS+x7NAy0nLTAKjhX4MOYR04N/RcGtZsSMOQhvRp0qdQ02lseizrjqxjXfw6cl25vHrRqyeNIT/5HdB8AItjFzO03VCePv/pUqf/6+hfLD5g9Yer6LO2tQm1Kjehioh8MUDks/6+jkIpVYKlsUvlkm8vkbVxa4uNS8lOkZum3yS9p/WWmLSYU152jjNHFsUsEqfLWabpY4/FepvVXl718imvL8uRJd9t/07Sc9NPON2CmAVij7TLCyte8A47mnVU+n7bV55d9myhJsoVB1fI/rT9J1337pTdMnvv7FKbLksSlRglf8T+IZsTNktCZoJ3+Pak7dJlUhcZNmuYdJnURQbPGCzxmfGF5s1z5cmM3TO8zaD/Xf7fQuNTc1LlyaVPij3SLp0iO0mnyE7yn2X/KbSekxm3ZpzYI+1yy8xbijXb7kndI9f+fK10ntRZZu+dXablud1u+fSvT+XTvz4ttjynyykvrHhB7JF2eXzJ44XGJWUnSc8pPWXs0rGSlpsmF067UB6Y90CZy/FPrT682vu93HJ0S7HxWY4sicuIK/P3/O94f8P73m154NiBClvPqUKbUC1VsgYOYNGrsGyCdT24GqG+jkYpVYScoBYwz5VHpiOz1E7x5e2eufew9shapt8wnVZ1y36W56nacnQLbeu3LVT7kevKLfPZvxUt/2zlC5tcyFv93iq19s8tbjYf3UzHsI4lXq5jYcxClhxcwh0d7qBtvbanFENidiJjlozhiR5P0PmszsXGZzmyOJJ1pNy2k1vc/HHwD85vdH6xmt6317/NV1u/4vLmlzM/Zj4/XPfDabtsjIhwy2+30DGsY4nN6KeDW9z8b83/CPYP5vEej/skhpJoE6pHlU3gYlbCV1fDv76AzoN9HY1S6gwWnRRNVFIUN7Wt3vdQdrgcLD+0nIuaXnRarqN2pkvJSeGqn64iy5nFNa2u8faHO13y8xFfdXc4U5WWwOklqquKcy6wbqm1PtLXkSilznDtw9pX++QNIMAvgP7N+mvy5lEvuB53dLiDYL9gRkWMOu3rN8Zo8nYKNIGrKmw26H4XxCyHozt8HY1SSqlK6N8R/+b3m37n7Npn+zoUdRKawJWz9BwHCcdyTj5hRYi4HWwBWgunlFLqb7EZW7ndBk5VLE3gytn1H67ghRnbfLPyWmdB+2th0zRwZPsmBqWUUkpVOE3gylnLsBD2JWb6LoDud0NOKkRN910MSimllKpQmsCVs+ZhIcQkZeGzs3tb9oX6rWHNJ+B2+yYGpZRSSlUoTeDKWcsGNcl2uEhIz/VNAMbAxWPg8EbY/K1vYlBKKaVUhdIErpw1D7MuBLnfl82oXW6Fs8+Hec9Bdqrv4lBKKaVUhdAErpy1bOBJ4JJ8mMDZbHDNBMhOhsWv+i4OpZRSSlUITeDKWeO6wQT4GfYnZfk4kC7Q415Y+wUcXO/bWJRSSilVrjSBK2f+fjbOqV/Tt02o+S59Fuo0hW9ugZQYX0ejlFJKqXKiCVwFaBEW4vsaOIAa9eC2H8GVC1NvhqxkX0eklFJKqXKgCVwFaBEWQkxSpu8uJVJQ+HlwyzeQsh+mDYHMRF9HpJRSSql/SBO4CtCiQU2y8lwc9dWlRIpq0QdunghHtsBn/SBus68jUkoppdQ/oAlcBWjhuZSIT+/IUFT76+DuOSBu+PIKWPEu5J1B8SmllFKqzDSBqwD5CVzMmdAPrqCm3eD+xVaN3Pzn4Z1O8McbkH7E15EppZRS6hRoAlcBmoRalxLZ58trwZWmdkO4/Se4Zx40joBFr8Bb7WHKzbDha0iN9XWESimllDoJf18HUBX5+9k4p15NYs7EBC5fs15wx8+QuAv++gb++g52z7fG1WsJjTpBeAfrJIjwDlC/FfgF+DZmpZRSSgGawFWYFg1C2Jd4hjWhlqRBG7jsebj0OTi6HfYshpgVEL8Nts+y+swB2AKg7tlQrznUbmJdoqRmPahR3/Pa85z/PjDEui+rUkoppcqdJnAVpHlYTVbvTUJEMJUhkTEGwttbj97/toY5siFxJyREW8ldSgykxkDSMuuaco4T1DD6BUJwKATVhqBaEFQHAmsVeF/bSvRCwq1nAHFZiWJgiOdRCwJrWq8DQsA/sMI/BqWUUqoy0ASugrRsEOK9lEh4nWBfh/P3BNSwbsnVuEvJ4525kJ1iJXPZycVfZ6dAbgbkZVjPaQchL916nZtuXWD4VHiTuyKJXUAw+AeDf1DxZ7+gkocH1LTK532ucXy8X+DxZ5t2E1VKKXXm0QSuguSfibr7aEblTeBOxj8IajeyHn9HXiZkHrUSPQwYG7gd1vC8LCvxy8sER/7rLM+4TKv2L/91doqVTDpzij+78v5ZGW0BnqQvsHByV+x9UOnj/AKtGk5jPOUs+IyVZNaoBzVCPZ+By6qNFLf12uZnLcMvwJNUBhx/7RcANn9rPpsfGD/rudgwfyvGylAbrJRS6qQ0gasg9qZ1AfgrNo0LWzfwcTRnqPym0notKm4dbreVxBVK7nKs5mFHtpUc5j8XTPqcudbDlXv8tfd9Djjzjo/LSjrBtLnAGXBHDrASusBaViLndlrJobEVSQYDrKTPmqFAwlcg8bT5FZ42/7Wf5733tb/18I73/LsRAcTzjJWoiljLz/9OGD9rO7id1rMrz9qWxrN+Yyvl4TkQMKVNY4q/x1PGkl4XLb+xeWptg624nblWjH6B1gPA5bAS8Pw4bAWTas9rjDWN25Ooi8uzGs90+fMYW4HPDM/nVnCbUqAsRcps8zses7g939k8zzj/AuvxL5z0u11WmST/WQp8RzwPm9/x30n+QUb+coyftb7835VfAPjXsJ7FfXx75x+k5B+o5H/uxlb48y60HU7iVO5+4/0ulfAZnGh9bpf1P8C7ASjhd1LSuJMGVOA7+jcPtFzOAosr8B2uqgdu//RuR+VxtyQfttJoAldB6ocE0jysJptiU3wdSvVms4Et2Nrh+oKItUP3JiwFnvPHO3MhJxWyU62dma3Ajt+7Q3VYy3HleZ4dnmGexMa7w3Udfy702mHtUHMzrJ2PX8DxHa0rr/Dy83ewVoDH48x/djuPT5+/M3M5PMmWJy6309qZeKfzjIMiO5YCO2hxWzv8/BNn8tn8PbWYtgIJQJGHUuWppKTO5nc8Ka34AEo42LAV+L0UORhxOaxWCbez5MUF1obgutb/ltx065GfNP9tZ8iBqS/514D/+u46qprAVaCIc0JZvTfJ12EoXzLm5CdfBNWCkLDTE8+ZTsSTxEmBWr4yHOGKFK7NKTHRKzCNt/anYM1W0ddFklewdpD5tbc2P0+ztN/xWkKMVdNo8z8eR9EaLbfbWnbBJm5vTVsJyXeJNTr5w+QkZc5PcD3fQ7/AAkl4fjzOAvG5i9cYGnO8JjS/Fs/t9PQZDSrQ7O8uULNrjvcvdTmO12wXqiksUpvq/aylwLbxbIv8GtrylF8LWPCz8JahpGEuqzxBta2uEccXVOR7UuS7VOZ4KGG7CSV/h4t8l/0Cj3/emMIxiMtK2HLSrHIE1bHKYCuH3f8/3iblsE19GYO3tcI3NIGrQBHnhDJ902Hi0rJpXLeGr8NR6syX34z6d+YzfoCfXq9QKVUt6Cl2FahrM+vyGJsOpPo2EKWUUkpVKZrAVaD2jWsT6GdjU2yqr0NRSimlVBWiCVwFCvL3o0OTOmzUBE4ppZRS5UgTuAoWcU4oWw6m4XTpmXJKKaWUKh+awFWwrs1CyXa42Bmf4etQlFJKKVVFaAJXwSLOCQXQfnBKKaWUKjeawFWwZvVrUj8kkI0H9IK+SimllCofmsBVMGMM3ZqFsnpfElIet+1QSimlVLWnCdxpcHn7hsQmZ7Pt8DFfh6KUUkqpKkATuNNgQIeG2Az8vtV390xTSimlVNWhCdxpEFYriF4tw5izNc7XoSillFKqCtAE7jS5ulMj9hzNZFd8uq9DUUoppVQlpwncaXJlx0YAzNFmVKWUUkr9Q5rAnSYN6wTTo3k9TeCUUkop9Y9pAncaXWVvRHTcMfYnZvo6FKWUUkpVYprAnUYDOzXGz2aYsjrG16EopZRSqhLTBO40ahJagxsimjJlTQyJGbm+DkcppZRSlZQmcKfZQ/1bk+d08/myvb4ORSmllFKVlCZwp1mrs2pxXZcmTF4VQ3Jmnq/DUUoppVQlpAmcD4zqfy7ZDhcTl+/zdShKKaWUqoQ0gfOBNg1rM7BTY75YvpcdR/TCvkoppZQ6NZrA+cgL13WgVlAAo6ZtICvP6etwlFJKKVWJaALnI+G1g3lnaAS7j2bwwvRtvg5HKaWUUpWIJnA+dFGbBozqfy4/rD/IF3pWqlJKKaXKyN/XAVR3j17Whp3x6bzyWzRH03N56qrzsNmMr8NSSiml1BlMa+B8zN/Pxke3def2C5rx6R97eeTbjaTo5UWUUkopdQJaA3cG8LMZXr7eTpPQGrw5byfLdyfyxBXtGHZ+M/y0Nk4ppZRSRRgR8XUMp1WPHj1k3bp1vg6jVDuOpPP89K2s2ZdM09AaDO5xNoN7nEPT0Bq+Dk0ppZRSp5kxZr2I9Cg2XBO4M4+IMC8qnimrY1i2KxGAtg1r0efcBnRrVo92jWrTskEIAX7aAq6UUkpVZVU2gTPGXAW8C/gBX4jIuBNNXxkSuIJik7OYtTmOlXsS+XNfMrlON2A1uzaoFUjDOsGE1w7irNrBnFU7iNpB/tQM8iMk0J+agX7UDPQnwM/g72fD32bw9zME+NnwsxkCbDb8/AwBNmu8zYDBgMF6bQwGMAZsxmrKNZ5pio43Rpt6lVJKqfJWJRM4Y4wfsBMYABwE1gLDRCSqtHkqWwJXUK7TxZ6ETHbEH2NPQibxx3JISM8l/lgOR9NzSfLxyQ9WcmclcwWTQYOVABYcn/8aCid/+S+LjjNFxucPKTp94WElT1M02fSOLzJfWeKhzPEWj4eTzFNaOQqvv5R5KZuyJt6nkp6XNZc3ZV1q+U5W5visZZbx8ylrjOX82VTEcVN5fyfKXmZVUfQAu2IE+tn45I7uFb6e0hK4yn4Sw/nAbhHZC2CM+Ra4Hig1gavMgvz96NCkDh2a1ClxvMstZOU5ycpzkZlrPWfluXC43DhcblxuweESXG7B6XZ7XlvPTpcbt4BgNeECuEUQ7zDrPZ7x1nsQCk6T/7rweM8fbrcUW1ZB+esV73vPM1LkfeHxFBl/KvPmjy/ydJJ4Sh5PKev8J2Upuq6yxFNWZZ1cKPtyy7zMcl532ZdX9mmtVZ984oqIsWzLO4XtUuZllu/yyrrAyluNcOarxHU0Z7xAf992Y6rsCVxTILbA+4NALx/F4nN+NkPt4ABqBwf4OhSllFJKVaDKnsCViTFmBDDC8zbDGLOjglfZAEis4HWcyapz+atz2UHLr+WvvuWvzmUHLX9Flr95SQMrewJ3CDinwPuzPcMKEZHPgM9OV1DGmHUltVdXF9W5/NW57KDl1/JX3/JX57KDlt8X5a/s16FYC7QxxrQ0xgQCtwAzfByTUkoppVSFqtQ1cCLiNMaMAuZiXUZkoohs83FYSimllFIVqlIncAAiMhuY7es4ijhtzbVnqOpc/upcdtDya/mrr+pcdtDyn/byV+rrwCmllFJKVUeVvQ+cUkoppVS1owlcOTLGXGWM2WGM2W2MedrX8VQ0Y8w5xpjFxpgoY8w2Y8yjnuH/Z4w5ZIzZ5HkM9HWsFcUYs98Ys8VTznWeYfWNMfONMbs8z/V8HWdFMMa0K7CNNxljjhljRlfl7W+MmWiMSTDGbC0wrMTtbSzvef4fbDbGdPNd5P9cKWV/wxiz3VO+X4wxoZ7hLYwx2QW+A5/4LPByUkr5S/2uG2Oe8Wz7HcaYK30TdfkppfzfFSj7fmPMJs/wKrX9T7Cv8+lvX5tQy4n5G7f1quyMMY2BxiKywRhTG1gP3AAMATJEZIIv4zsdjDH7gR4iklhg2OtAsoiM8yTy9UTkKV/FeDp4vv+HsC6kfTdVdPsbY/oCGcDXImL3DCtxe3t25g8DA7E+l3dFpNJeaLyUsl8BLPKcUDYewFP2FsCs/OmqglLK/3+U8F03xnQAvsG6W1ATYAHQVkRcpzXoclRS+YuMfxNIE5GXqtr2P8G+7i58+NvXGrjy472tl4jkAfm39aqyRCRORDZ4XqcD0Vh3x6jurgcmeV5PwvqhV3WXAXtEJMbXgVQkEfkDSC4yuLTtfT3Wzk5EZDUQ6tkRVEollV1E5omI0/N2Nda1OKukUrZ9aa4HvhWRXBHZB+zG2kdUWicqvzHGYB24f3NagzpNTrCv8+lvXxO48lPSbb2qTTLjOeLqCqzxDBrlqTqeWFWbED0EmGeMWW+sO34ANBSROM/rI0BD34R2Wt1C4X/e1WX7Q+nbu7r9T7gHmFPgfUtjzEZjzFJjzMW+Cuo0KOm7Xt22/cVAvIjsKjCsSm7/Ivs6n/72NYFT/5gxphbwEzBaRI4BHwOtgQggDnjTd9FVuItEpBtwNfCQp5nBS6w+ClW6n4KxLqI9CPjBM6g6bf9CqsP2Lokx5lnACUz1DIoDmolIV2AMMM0YU8dX8VWgavtdL2IYhQ/gquT2L2Ff5+WL374mcOWnTLf1qmqMMQFYX+ipIvIzgIjEi4hLRNzA51TypoMTEZFDnucE4BesssbnV5d7nhN8F+FpcTWwQUTioXptf4/Stne1+J9gjLkLuBa4zbMTw9N0mOR5vR7YA7T1WZAV5ATf9Wqx7QGMMf7Av4Dv8odVxe1f0r4OH//2NYErP9Xutl6efg9fAtEi8laB4QXb+m8EthadtyowxoR4OrRijAkBrsAq6wxguGey4cB030R42hQ6+q4u27+A0rb3DOBOzxlpF2B18I4raQGVlTHmKuBJYJCIZBUYfpbnxBaMMa2ANsBe30RZcU7wXZ8B3GKMCTLGtMQq/5+nO77T5HJgu4gczB9Q1bZ/afs6fP3bFxF9lNMD64yTnVhHG8/6Op7TUN6LsKqMNwObPI+BwGRgi2f4DKyzd3webwWUvxXwl+exLX+bA2HAQmAX1tln9X0dawV+BiFAElC3wLAqu/2xEtU4wIHVr+Xe0rY3YIAPPf8PtmCdrezzMpRz2Xdj9fXJ//1/4pn2Js9vYhOwAbjO1/FXUPlL/a4Dz3q2/Q7gal/HXxHl9wyPBB4sMm2V2v4n2Nf59LevlxFRSimllKpktAlVKaWUUqqS0QROKaWUUqqS0QROKaWUUqqS0QROKaWUUqqS0QROKaWUUqqS0QROKeUzxhjx3AQ7//0TnhuEl8eyI40xN5fHsk6ynsHGmGhjzOIiw5sYY370vI7w3OC6vNYZaoz5d0nrUkpVD5rAKaV8KRf4lzGmga8DKchzdfmyuhe4X0T6FxwoIodFJD+BjMC6blR5xRAKeBO4IutSSlUDmsAppXzJCXwGPFZ0RNEaNGNMhue5n+cG2dONMXuNMeOMMbcZY/40xmwxxrQusJjLjTHrjDE7jTHXeub3M8a8YYxZ67kJ+QMFlrvMGDMDiCohnmGe5W81xoz3DHse6yKfXxpj3igyfQvPtIHAS8BQY8wmY8xQz108Jnpi3miMud4zz13GmBnGmEXAQmNMLWPMQmPMBs+6r/csfhzQ2rO8N/LX5VlGsDHmK8/0G40x/Qss+2djzO/GmF3GmNcLfB6Rnli3GGOKbQul1JnnVI4ylVKqInwIbM5PKMqoC9AeSMa6Rc8XInK+MeZR4GFgtGe6Flj3p2wNLDbGnAvciXVrm57GmCBghTFmnmf6boBdRPYVXJkxpgkwHugOpADzjDE3iMhLxphLgSdEZF1JgYpInifR6yEiozzLew1YJCL3GGNCgT+NMQsKxNBZRJI9tXA3isgxTy3lak+C+bQnzgjP8loUWOVD1mqlkzHmPE+s+fehjAC6YtV87jDGvA+EA01FxO5ZVugJPnel1BlCa+CUUj4lIseAr4FHTmG2tSISJyK5WLeryU/AtmAlbfm+FxG3iOzCSvTOw7pn7Z3GmE3AGqzb4bTxTP9n0eTNoyewRESOiogTmAr0PYV4i7oCeNoTwxIgGGjmGTdfRJI9rw3wmjFmM9atepoCDU+y7IuAKQAish2I4fiNxBeKSJqI5GDVMjbH+lxaGWPeN9a9TY/9g3IppU4TrYFTSp0J3sG6Z+JXBYY58RxkGmNsQGCBcbkFXrsLvHdT+P9a0XsFClZS9LCIzC04whjTD8j8O8H/DQa4SUR2FImhV5EYbgPOArqLiMMYsx8r2fu7Cn5uLsBfRFKMMV2AK4EHgSHAPf9gHUqp00Br4JRSPuepcfoe64SAfPuxmiwBBgEBf2PRg40xNk+/uFZYNxafC4w0xgQAGGPaGmNCTrKcP4FLjDENjDF+wDBg6SnEkQ7ULvB+LvCwMcZ4Yuhaynx1gQRP8tYfq8aspOUVtAwr8cPTdNoMq9wl8jTN2kTkJ+C/WE24SqkznCZwSqkzxZtAwbNRP8dKmv4CevP3ascOYCVfc4AHPU2HX2A1H27wdPz/lJO0RohIHFa/s8XAX8B6EZl+CnEsBjrkn8QAvIyVkG42xmzzvC/JVKCHMWYLVt+97Z54krD67m0tevIE8BFg88zzHXCXp6m5NE2BJZ7m3CnAM6dQLqWUjxiRoi0MSimllFLqTKY1cEoppZRSlYwmcEoppZRSlYwmcEoppZRSlYwmcEoppZRSlYwmcEoppZRSlYwmcEoppZRSlYwmcEoppZRSlYwmcEoppZRSlcz/A8pvXjtAWamEAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#**ANSWER FOR 2ND QUESTUON**"
      ],
      "metadata": {
        "id": "z18wxZQguzao"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!gdown --id 1WGdO18u8FIlmUsLsak9sZ7s7Mw7mkKCz"
      ],
      "metadata": {
        "id": "nu-ONgwIuyd6",
        "outputId": "34a819c2-5d97-4ed7-fd48-95862f747d88",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/usr/local/lib/python3.9/dist-packages/gdown/cli.py:127: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.\n",
            "  warnings.warn(\n",
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1WGdO18u8FIlmUsLsak9sZ7s7Mw7mkKCz\n",
            "To: /content/insurance.csv\n",
            "100% 55.6k/55.6k [00:00<00:00, 47.4MB/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "df = pd.read_csv(\"insurance.csv\")\n",
        "df.head()"
      ],
      "metadata": {
        "id": "DBensJMou-_o",
        "outputId": "9647254e-8811-42a8-f747-ab9ae0029803",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   age     sex     bmi  children smoker     region      charges\n",
              "0   19  female  27.900         0    yes  southwest  16884.92400\n",
              "1   18    male  33.770         1     no  southeast   1725.55230\n",
              "2   28    male  33.000         3     no  southeast   4449.46200\n",
              "3   33    male  22.705         0     no  northwest  21984.47061\n",
              "4   32    male  28.880         0     no  northwest   3866.85520"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6451e19b-7451-46e0-b625-dbd3b5511eed\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>sex</th>\n",
              "      <th>bmi</th>\n",
              "      <th>children</th>\n",
              "      <th>smoker</th>\n",
              "      <th>region</th>\n",
              "      <th>charges</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>19</td>\n",
              "      <td>female</td>\n",
              "      <td>27.900</td>\n",
              "      <td>0</td>\n",
              "      <td>yes</td>\n",
              "      <td>southwest</td>\n",
              "      <td>16884.92400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>18</td>\n",
              "      <td>male</td>\n",
              "      <td>33.770</td>\n",
              "      <td>1</td>\n",
              "      <td>no</td>\n",
              "      <td>southeast</td>\n",
              "      <td>1725.55230</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>28</td>\n",
              "      <td>male</td>\n",
              "      <td>33.000</td>\n",
              "      <td>3</td>\n",
              "      <td>no</td>\n",
              "      <td>southeast</td>\n",
              "      <td>4449.46200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>33</td>\n",
              "      <td>male</td>\n",
              "      <td>22.705</td>\n",
              "      <td>0</td>\n",
              "      <td>no</td>\n",
              "      <td>northwest</td>\n",
              "      <td>21984.47061</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>32</td>\n",
              "      <td>male</td>\n",
              "      <td>28.880</td>\n",
              "      <td>0</td>\n",
              "      <td>no</td>\n",
              "      <td>northwest</td>\n",
              "      <td>3866.85520</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6451e19b-7451-46e0-b625-dbd3b5511eed')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6451e19b-7451-46e0-b625-dbd3b5511eed button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6451e19b-7451-46e0-b625-dbd3b5511eed');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Changing Text to numerical values\n",
        "# sex \n",
        "one_hot = pd.get_dummies(df[\"sex\"])\n",
        "df = df.join(one_hot)\n",
        "\n",
        "# smoker\n",
        "one_hot = pd.get_dummies(df[\"smoker\"])\n",
        "df = df.join(one_hot)\n",
        "\n",
        "# region\n",
        "one_hot = pd.get_dummies(df[\"region\"])\n",
        "df = df.join(one_hot)\n",
        "\n",
        "# Drop Unwanded columns\n",
        "df = df.drop([\"sex\", \"smoker\", \"region\"], axis=1)\n",
        "df.head()\n",
        "     "
      ],
      "metadata": {
        "id": "DzBmS1x-vQEd",
        "outputId": "7368e498-6169-4b4d-caa4-a50905cf0c4b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   age     bmi  children      charges  female  male  no  yes  northeast  \\\n",
              "0   19  27.900         0  16884.92400       1     0   0    1          0   \n",
              "1   18  33.770         1   1725.55230       0     1   1    0          0   \n",
              "2   28  33.000         3   4449.46200       0     1   1    0          0   \n",
              "3   33  22.705         0  21984.47061       0     1   1    0          0   \n",
              "4   32  28.880         0   3866.85520       0     1   1    0          0   \n",
              "\n",
              "   northwest  southeast  southwest  \n",
              "0          0          0          1  \n",
              "1          0          1          0  \n",
              "2          0          1          0  \n",
              "3          1          0          0  \n",
              "4          1          0          0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-c3ff1468-63a8-4ec9-92fb-e06abcb4df77\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>bmi</th>\n",
              "      <th>children</th>\n",
              "      <th>charges</th>\n",
              "      <th>female</th>\n",
              "      <th>male</th>\n",
              "      <th>no</th>\n",
              "      <th>yes</th>\n",
              "      <th>northeast</th>\n",
              "      <th>northwest</th>\n",
              "      <th>southeast</th>\n",
              "      <th>southwest</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>19</td>\n",
              "      <td>27.900</td>\n",
              "      <td>0</td>\n",
              "      <td>16884.92400</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>18</td>\n",
              "      <td>33.770</td>\n",
              "      <td>1</td>\n",
              "      <td>1725.55230</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>28</td>\n",
              "      <td>33.000</td>\n",
              "      <td>3</td>\n",
              "      <td>4449.46200</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>33</td>\n",
              "      <td>22.705</td>\n",
              "      <td>0</td>\n",
              "      <td>21984.47061</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>32</td>\n",
              "      <td>28.880</td>\n",
              "      <td>0</td>\n",
              "      <td>3866.85520</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c3ff1468-63a8-4ec9-92fb-e06abcb4df77')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-c3ff1468-63a8-4ec9-92fb-e06abcb4df77 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-c3ff1468-63a8-4ec9-92fb-e06abcb4df77');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df = df.sample(1000, random_state=42)\n",
        "X_df = df[\"bmi\"].to_numpy()\n",
        "y_df = df[\"charges\"].to_numpy()\n",
        "     \n",
        "\n",
        "# Standardization\n",
        "std = X_df.std() \n",
        "mean = X_df.mean()\n",
        "X_df = (X_df - mean) / std\n",
        "\n",
        "std = y_df.std() \n",
        "mean = y_df.mean()\n",
        "y_df = (y_df - mean) / std\n",
        "X_df.shape"
      ],
      "metadata": {
        "id": "-CG9V6Vcv7ew",
        "outputId": "88d6ef63-b8c6-4603-f966-f9fe1068b55a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1000,)"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "X_df = np.expand_dims(X_df, axis=1)\n",
        "X_df.shape"
      ],
      "metadata": {
        "id": "j7NvceeRwBpx",
        "outputId": "f4a929df-33cb-4b5b-89ca-2843820e813d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1000, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "plt.plot(X_df, y_df, 'b.')\n",
        "plt.xlabel('x')\n",
        "plt.ylabel('y', rotation=0)"
      ],
      "metadata": {
        "id": "kv2X2L-swFGw",
        "outputId": "32e3e150-ce65-443e-a18f-d681f4545f7d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        }
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, 'y')"
            ]
          },
          "metadata": {},
          "execution_count": 23
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEGCAYAAACAd+UpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAA8R0lEQVR4nO2de5BcV33nv6cfMzIbUi4GZRUgimOcgBUm0diDlslDDI8oNoFkgiopiDdybGMxYJuYymYqWopaEbNSAokzwSbZGYEUqZYllYp4G8o24CkrmSH22CPjYMeEkFghrAoYSsWmFj1m5uwfv/ntPff0OffV9/a9ffv3qeqa6e57zzn3dvfv/M7vdZTWGoIgCEI9aZQ9AEEQBKE4RMgLgiDUGBHygiAINUaEvCAIQo0RIS8IglBjWmUPwOb5z3++vuKKK8oehiAIQl/x2GOPfUdrvdV+vXJC/oorrsDy8nLZwxAEQegrlFLPul4Xc40gCEKNESEvCIJQY0TIC4Ig1BgR8oIgCDVGhLwgCEKNESEvCIJQY0TIC4JQG5aWgMOH6a9AVC5OXhAEIQtLS8BrXgNcvAgMDQFf+AIwMVH2qMpHNHlBEGrBwgIJ+PV1+ruwUPaIqoEIeUEQasHkJGnwzSb9nZwse0TVQMw1giDUgokJMtEsLJCAF1MN0RMhr5RqAlgG8G9a69f3ok9BEAaPiQkR7ja9Mtf8NoCne9SXIAiCsEnhQl4p9SIAvwTgQ0X3JQiCIITphSY/C2AGwIbvAKXUfqXUslJq+dvf/nYPhiQIgjAYFCrklVKvB/AtrfVjUcdpree11uNa6/GtWztq3guCIAgZKVqT/1kAv6yU+hcAfwng1Uqp/1lwn4IgCMImhQp5rfUBrfWLtNZXAHgTgC9qrf9zkX0KgiAIAZIMJQgJkJooQr/Ss2QorfUCgIVe9ScIeSE1UYR+RjR5QYhBaqII/YwIeUGIQWqiCP2M1K4RhBikJorQz4iQF4QESE0UoV8Rc40gCEKNESEvCIJQY0TIC4Ig1BgR8oIgCDVGhLwgCEKNESEvCIJQY0TIC4Ig1BgR8oIgCDVGhLwgCEKNESEvCDkgpYiFqiJlDQShS6QUsVBlRJMXhC6RUsRClREhLwhdIqWIhSoj5hpB6BIpRSxUGRHygpADUopYqCpirhEEQagxIuQFQRBqjAh5QRCEGiNCXhgoJGlJGDTE8SoMDJK0JAwioskLA4MkLQmDiAh5YWCQpCVhEBFzjTAwSNKSMIiIkBcGCklaEgYNMdcIgiDUmEKFvFJqi1LqEaXUE0qpryil3lNkf4IgCEKYos01FwC8Wmv970qpNoC/UUp9Tmv9pYL7FQRBEFCwJq+Jf9982t586CL7FIR+QJKyhF5RuONVKdUE8BiAqwB8UGv9d45j9gPYDwDbt28vekiCUCqSlCX0ksIdr1rrda31TgAvArBLKfUyxzHzWutxrfX41q1bix6SIJRKVFJWrzV8WVHUn56FUGqtzymlHgJwHYC/71W/glA1OCmLNXlOyuqFhr+0RJPKyAiwsgIcOwasrfn74+Mlr6B/KVTIK6W2Ari0KeAvA/ALAP6wyD4Foer4krJcGn5WweoSzjyJXLgAbGyEj3f1J2alelC0Jv/DAI5v2uUbAP5Ka/2ZgvsUhMz0SnN1JWX5NPy0+IQzTyK2gFfK3V+ek45QHoUKea31lwGMFdmHIORF2ZprXmUXfMKZJxHW5BsNoNUCbr4Z2LevuElHKBcpayD0NVGad1qtvCjN1Wc6cY0tj7ILPuFsTiIjI8Dqqv/e8PhmZ6OPE6qPCHmhb4nSvLNo5Wk11ySTiGscQLErhqgVQZJJpOwVjZAvIuSFviVK886ilbNwPHEivu+4CYYFrGscZ84A588DWhdn6+5mRSC2+HohBcqEviWqPnw3teOPHweOHCEh7osf98W6s/B/97vp78hIeBwjI8DRoyTgAbKJV83WLXX364Vo8kLfMjFBNuOTJ4G9ezvNElmcmEm1WJ9pxz5/dTU8joUFeg+gqJabbqqeluy6d1mijiTGvhqIkBf6lqUl4M47SZieOgWMjsY7NuNIapf3TSKu823Tifn+vn2pLrlnmGPOYqPP264vE0Z2RMgLfYtP6+5GwKRZAbjs3nHn9+PuVFls9HkndokjODsi5IW+JanJ5MSJdEK12zDGuPP7bXeqLPHyecbYiyO4O0TIC31LEpNJsxlfn0WIJsvqI88ViyRldYfSulrl3cfHx/Xy8nLZwxD6HLbhnjlDkTLr6yTw77oLOHAgv/Y5qSguuaibPvrFrFMkdbsXRVyPUuoxrfW4/bpo8kItYZPI0hKFROapBdqFvpSikMhGAxgezme1EGeHrpvQi6PfTFxR9NrHIHHyQmXJUuvcPofNBnfdFf9jStqfXeiLF8MbG5314bMSV3PejMVPcn/S3kupM18cUZ9tEYgmL1SSJNqOrc36zsk7ld9V6Iv/5rVaiLJDp3VEptUcJZqlWHrtYxAhL1SSOEHmEkTdRGGkOddV6Ctvm3yU4zKtkEh7XySapVh6HUYrQl6oJHGCzCWIutGQXOdG2b2z2IjT2tF9fURl+rpIe18kmqV4euljkOgaobLElRF2mRS6cUia5wL5mizm54HbbiOzTlLnrO9asmagprkvg+bYrQMSXSP0Hba2Ywse15K3Gw3JPPfw4e5NFmaY5e23U6w+QLb8buzoWStsphl/naJZBh0R8kJfEOVUjTuvyBo2vn7m50mwr6+TQ5aLkgH0PK0d3czarbM5RVYQ+SNCXugLsmivvaphY/czO0umGdbcNzYoEUtr+nvvvens6K6s3X6rf5MEieopBhHyQl+QRXvtNkokqcnC7ufkyc7NshsN4JZb3Hup+vpmQW5m7fJ1HDhQPwEoUT3FIEJe6AuSatbmcr9Is0ZUP3v3Uulj3v0JIKG/fXu2SaaIrN0qUmczVJlIdI1QG3z7qeZt1kjSz9IS2dHzKo42KLbqQbnOIpDoGqH2uJb7RZg1ovrhcgCTk8Cf/zmZZ0RoJUeievJHhLxQG5Is99NoivPzQcLR/v3uflotsplzjZcsEUBxiENS6AYR8kLlsZOUorJQo+z2aYTl/Dzw1rfS/w88QH/37w/GMjsLrKzQptxHjpDN/MYb4x2HSa/FpK4OSTHN9AYR8kKlMQUzl/TVOsgaBcKCIkpzTiMsT57sfD46Gp4kbryR2uL2gOiVhHktrRZdx/p68oJodXJIyuqkd4iQFyrL0hJw8GBQ7dHkwgVybJpRJ3GCYnKShOvGBv2NEpZ79wYaPD+3Jwmgc1PuKBu8eb5ZptiecGwNt9cFrXpBXVcnVUSEvFBJ7I05bJpN+ptWUHAwWVxQ2egoTQRra9TXygowNtYp1MfGOguF+Uw0IyNhW76pyfOE00255KIowqxSx9VJVREhL1QS1vS4Tvv4OPDEE8ClS/T83ntJEKeJH19YIKHKwpUnBZcQW1gIJoL1dWBuDtiyhWzxXFIYAO68k/o/dYrGE+cHuOMO4PRpmhRGR939ZtFwi7JvF2VWqePqpKoUKuSVUj8C4ASA/whAA5jXWv9pkX0K9cDW9GZn6XVbKKQRFL5ywi4hxsdyQhObVVZXgz1iXUXM7DGaQvvCBeDuu6mtU6eoL3u/2SwabpQg7lb4F2lWkXDJ3lC0Jr8G4He01o8rpZ4L4DGl1INa66cK7lcogTy1yagqk/ZxUREs5nuuNn3VJvnYEycogsY2qwCdAnlkpFPYmsdwoTJzm8C0EUIufII4Dy1czCo1QGvdsweATwL4hahjrr32Wi30H4uLWl92mdbNJv1dXOyPcZjHDw9rPT3dec7iotaHDrnbMt+bntZaKdL7m0163Txmbi48trk5f7t5XPOhQ/SaPZ4s7ecxTqFYACxrl9x1vVjEA8AVAM4A+EHHe/sBLANY3r59e8G3QiiCvARKFEmETZZxsIAeGso+SS0u0iTBxp2hIbcQ9wn8xcXuhKnr3KpMvEJvKFXIA/gBAI8BeGPcsaLJ9ydFC5Sk7buOy3Ny8LVlavFKaT01FT1eu7/p6WLun2jhg4NPyBceXaOUagM4CeAjWuuPFd2fUA5FR0skdQDa4wCS2aWTlkTwbTl49GgQjTM0BGzbFj1euz+gGAenODeFoqNrFIAPA3haa313kX0J5VOkQElblyYqAsa3paBvkuJjzpxxt8WhmQBl5d50E8XQR4V3uiYj1/GS+i90S9Ga/M8C+E0ATyqlTm++9l+11p8tuF+hZmStS+OKgHnb2+icsbEgzp3PsUMazXabTUpiAsKC2O6DNwaJW9nYk6J9vKT+C3lQqJDXWv8NAFVkH8LgYCYMmc/5tahQSM44veOOoCQBb8kXFdJotgsAt95Km3/YYZ0cbmmPt5udqCT1X8gDyXgV+oYozTbKnMPC8/BhyphleO9VpfwmIJ+W7oLNLceP56N1S4y6kAci5PuIsu2zZfcfpdkmMY9MTgLtdri42Ac+EJQpSOLIzaPCZVIk9V/IAxHyfULZ9tmy+wfCmm2zGd6sg80xUbCTlM0qaTbVNssEuM4vSuuW6BihW0TI9wll22fL7h8I276PHaPNOo4dI7v62hqZX5QibT1qww4WzuZWfUmLgE1OBiuBY8eAhx6qbzlgoR5ECnml1O8D+K7Wenbz+X8H8C0tRcZ6Ttn22TL6d2nNrI2vrXXWZee/Fy/SeXYYpLkSmZ3tjKyJE8wLC2GbvstkJMJdqBpxmvxRAB8DMKuUagB4E4BdhY9K6KBsTbHX/S8tAa96FVVuBCjZiAWqvceq1iR8WdADwOOPUxu+Er4nT6Zfmbhs+uIMFapOpJDXWv+LUmpVKTUGKhe8orVe7c3QBJuyNcVe9c87QrEwBUiInzjhTlwCgPe9D/jEJ4LjH32UNHdfvPzevVTuN83KJKtNPyllO7aFepLEJv8hAL8FYBtIsxeEwjB3hDI182aTbOBra+7EpV27gE99yr+tnmsl4tq0I46iJroqOLaFepJEyH8cwO8DaAP4jWKHUw6iQVUH145Q11xD7x05El0LZng42C6w0fDHy/uel0kVHNtCPYkV8lrri0qphwCc01qv92BMPUU0qGph29uvuYbMIkDyWjAjI9Gx71WkbMe6UF+UNtfErgPI4fo4gF/TWv9j0QMaHx/Xy8vLRXfz/zl8GHj3u0mDajaBu+7qrF8i9AZzw+uVlfCOTF/4Ah1T5xWXrCiFblBKPaa1Hrdfjwuh3AHgMwA+3gsBXwaiQVUDe0V1440k4E3zxYED9RZ+VTIfCfUhLrrmKQBX9mgspVB2aKJA2DZpoL6Tr2jsQi+RjFeIBlUFXIXA9u2rnzAUH5DQa0TIC5XAt6KqmwCUKBqh14iQFyrDIKyoxAck9BoR8gNC2XbgpP2XPc6iER+Q0GtEyA8AZduBk/Zf9jh7RdSKpe6TnNB7GmUPQCgelx24CLh0L9d4T9t/r8ZZVXiSe/e76a99HwUhC6LJDwB52oF9mmbWrfmKGmfceKuIOGWFIhAhPwAktQPHCUTeNOPSpc6NOXxauK9qpGuzjrzt1f1m/hGnrFAEIuQHhLjIlSQC8cSJIFHJ3pjDFlAjI53tHTgQ30+eETb9phmLU1YoArHJCwDysYffeCNw660kqFZX/Zp9r+zuPPE0m/2jGU9MlFe+wedTEfob0eQFAPGmAv7ht9tBTXeuDmlr5/v2+dvrpUmibM24n/wB/WbaEpIjQl4A4BaIZlVI3g+12QTe+tbwrkgu7fzAAX8Gay8Fby8SrFzCvN+EZr+ZtoTkiJCvML3WBE2BaAqpRiO8afb27Z0bdri0c5+AzTNO3Cdge7kXrUuY95vQFKdvfREhX1F8wqMbAZbmXFNIaU2CXqn4DTuStJ0lDNPXjn080FsN2ifM+01olm3aEopDhHxF8TkoswqwtALUFlKzs9G7LSU1i0SNI63267tHvdSgo1Yx/SY0B6F20CAiQr6iuIRHNyaAtOcWJaSixpFW+/Ud30sNOuo+uYRmPzljhXogQr6i+IRHEgFmOkxZ+85iPjA17CefDNri17IIqqhxpJ1YfMf3WoPOYxUjCEURu8dr1x0odRTA6wF8S2v9srjje73Ha78R52gESJBcuECO0kYDGB7OtkcqCyWzrVaLbPMcRplFUA2qNiv7CQtFkmmP15z4CwD3AjjRg75qj601uvZGvXgxiITZ2Mi+RyqbVsy2Ll2i/7XObvMeVNtvvzljhXpQuJDXWj+slLqi6H4GFd/eqKb2nVWgsFCK0uSrKqiquFroR2es0P8Ubq4BgE0h/xmfuUYptR/AfgDYvn37tc8++2zhY6oLvjBC2yafVaC47PvcfjfFzoqkLrbvKk5UQnUp01wTi9Z6HsA8QDb5kofTV/i0wyS7LyWZBKISmrjWyblz1N4LXgDMzND7aYVsngKt3xKRXNRlohLKpxJCXuiONDZulzM1i2OW2zl/nuzzzH33Abfckk7I5i3Q6mD7rsJEJSuJeiBCvoZEReCcOeN2zJ44ARw7FgjGhx5KlohkW/vYMZtGyCYRaGkETh1s32VPVLKSqA+FC3ml1EcBTAJ4vlLqGwD+m9b6w0X3O6jEpfo3m+Q81TrsmD17lrR7gP6ateJdsBCyNfl2m4qX7duXXMgmqYCZVuDkEcFTpiZb9kRVhZWEkA+9iK55c9F9DApJhI7547xwATh4ELjyynAdmvFx4JprgLExssmPjAAftqbdxx+n/qJs9SyEbJt8Er+Ary3XtZUhcKqgyZYZalr2SkLIDzHX9AlJhY4d9vj5z5N23WwG2vvyMmWwcj14ttEDFB6pNR3zmtdECzdXzD7Xj8kzdt4UOK0WmZyiJqAkxE2Yg67Jlr2SEPJDhHyfkFTo8I/z4EES8BsbFNN+663A178evGYX9GLTzZVX0nHmMUnCJIHiNF++phMngKNHgSNHgOPHs/eRZMIUTXZwk9bqhgj5PiGp0GHBu3MnOU+B8C5Op05FF/T63d8NNggxj7E1X1+mLU9CJ05k0wJ9GvbEBL2+vt69dp1kwhRNVqgLIuT7ABZ8ceV+7fBIpchMMzvrLt4FuNsdHY0W6NyGK9OWTSpHj9J7abR6c/zNJnDvvcD+/cH7eWnXSdsRTVaoAyLkE1D17E1feKTW9FhdDR83OUk2+dtuo2M5Tt50mJp9uJy5e/eGBaUZUXPmDJlU0mrcCwvBBLWxQeMbHQ2PKw/tWrR0YZAQIR9D2VEWcaYFc3ytVtjB2mjQY2Sk87i1NWoTIMEaJYhdztxTp9wrC9b8jx9Pr3FPTtL4zRh+e1x5adeipQuDQqPsAVQdl5DNCpcBWFpKfg4L2GaTHhxZ4hrf2hpw883Ae99LoYy8N+udd5KN3LwOFqQAHRcliFnzfe1r6Vh2yq6uuitb8vG33kq2+qRMTJCJptUKMnEH0eEpCHkimnwMedmBs64IzMiSY8c6I0vs8e3bR68fPhxo9C6budY0KbDtO0ly0cGDbsetD9bmjx+P9ycw+/d3+gSKIE39nqIp2hwo5QkGGxHyMeRlv+0m7pojS9jEYp7vG59L+JtZqDymNNeU5l7Ydvzbb6cJJ86vwO0WKYyi6vdUyeeSh3Au29wolI8I+QTkIXS6XRG4zjeFgL3DkE8g2/btOFzCN05ALy2RWanZpPeVImHvi73vtSDiCci1sUpVygfkdU8GPalLECHfM7pdEdjnA/FCoNvJKWlkj1kX53WvAz77WRIqrRbZ5cfGgth7V8aqKYjOn4+vm9Mtrs1QXBNnL4Shb/LPSzhLUpcgQr4AohJ6smjPrvMPH05euTGr3dkMafRF4JjCaH0d+MQnOttZXSWb/MqKO2OVo2q4ts6RIzQxmDHyeWJOmOfOAadPU0go0HvTRlJzW1bhLOGiArTWlXpce+21umwWF7U+dIj+Zjn3ssu0bjbpb9o2kp4fdxy/32hQtHyjET8e+7rn5jjSnh5zc53HcT9KhY8FtB4e1npoKBjj9DT9D9DfQ4eCvqenw+e229nufxrsexg1vjRtmvew2+9S1nOFwQPAsnbIVNHkLbq1hZ44EZTfzbLMTlujhjV1uzBYErtzXO2Z1dUgZLLRoOeu+zM7S1UsH3ssiL1vNoHrrwc+/Wl3Vqytne7bB3zoQ+RcBugcvqaitNCorN002rO5YjJLQszOhp+n/S5JLL+QByLkLbqxhS4tUZgj11dvNrt3sI6MkGnGJeT4uWtSirI781ijas+wYB0eDgs9+/6cOBGESgJBFUsA2LYtOsLHNkV98IPA299O42236dqLNJ/ERSClKcVw8SJdO2frXrwInDwpTk+hfETIW3RjC+UwR4B+8Dff3J2D1dYMXULONylNTJAmefIkFSu7/PKw4EqixfrsuXxcs0l1582JpNUKQiV9QpOjR+zJa3Q0yHjVmmz43QjJOCdqkgikOMz72GjQ+JWi69+7N11egSAUgQh5i24cVS7NMOsYOKEpTsj5JqWlpWCC4BIEpknHPm9sLGiTE6rMsZhjM5OzlpfDseZm0hPQeR+XloKSwXYBs4WFwPnKZp+0E67PdOJbBXRrErHvY1yxt6oiCVM1xmWoL/NRBcdrN+TpLEvqhJ2b03rPHvrL/ZtORKXI8Wo7X/nYubn0zuJDh4L2Gw3q32x3elrrVov6HhryO2lNB6fretPcT/P8djtwOmd1oial3x2k3QYLCNUA4ngtlqjEpKy4VhWuxCPWWBcWyFSwttZZrIzt5GYoZJoVg41ZTKzVopIHZhKPufcr2+63b6f/+XU2a7CG3q35xDSdaE2rC7uPKLJqs/3uIJWEqXojQj4HiszYNAVIXF13s8QwEOwG9eCD0c7gpH4I0xSyshIW1owZXWQzOUkTAptiGo1wrXv7el19RwngONNJFIOc/i8JU/VGhHwO5KEJpd2k24yAMR2hrMnbu0GdP0/vvfOd7iidOD+Ea0MSFuRra4G9/+hRt4AfG6N2b7oJmJsLjuFa93H3JokA7safMsjarCRM1RsR8jlgC9q0G02n3aQ7KgIG6Pyxzs4GG4Tccw8wNRVdAsE14dhx97bJZWSETDYcXWTCMfYATTxpa82nEcBZTSeDrs32u8lJiMBlqC/z0a+OV3Y2Dg+nd2AdOhTOTI1yEmZx8plO0jgnpM8J58qgHRqia2bHrf2e716kvYY0jsFByjDtt/EKxQJxvOaHS9PlEEBXOeA4RkbCmakjI/5jk2hc9vjSaKlRcfdm/L5p62bHLYdSvva1pNVze74aPK5Yed81JzEnJF0RdVtbqArk7UOQEMr6IkI+JVE/rqxL/tXVwMatVDI7ddrxJbW5Rl2DTwja53CkDZ+TZpw+kgjgJGadfnKwRgnePH0I/XRPhPSIkN8kqSYT9ePK6sAaGQls3FpHa/JxRGniUQLXHHPaa8hyThohlfSzSTLJRvVbJW02TvCa12qXb057HYPsdB4IXDacMh9l2OTT2nzztg+nscnHkTaxpaxEmLyqbbqOt++5q2qmz+dQlYSgJH4U9gOZlT6zJLZV7dqFbEBs8kRU5EjS6I087cMA1TQHyFTT7ebVabVq37W7kq7y1HKTjjOtlmmvWOzPYXY22FzcLN+QVZv13Zdu75dZYE4p9+qO/UBcy//iRaoGmrYKqoRQ1hyX5C/zUaQm30stLmlEi12zfWam+77T4CslYL6WRDtkrXJ6Ol9NMOvKif+fmgrKKDQaVGrBF/GTlwacx/dpcZHG3mzS+NvtoJ6/ydwcXRNHNLXbwXeJy0kIgwE8mnzhQhvAdQCeAfA1AL8Xd3yRQj5K8OYdjpb0h75rV1jI79jR+7A4+9rt+7RnT/SEtbhI4ZK2cMnrniZpxwzxtGvXmHVyzHo209PJN/hwvef7Pk1PBxNL2n7Ma7E3Ymm13BNTo0HXy5MC1yuank5/H4X+pRQhD6AJ4J8AXAlgCMATAHZEnVOGJl9kf3H2Yf5Rmo8kuzgVPe40mvyhQ2GBxAImy71OW5CMjzX9Gq6HUiQEXQXMhoez+WF8q6ChoaDfdju9zdycPOzvhTnB2r6cqHue5Lsvk0B/4xPyRdvkdwH4mtb66wCglPpLAL8C4KmC+3WSxvaYhw06zj58441BfLyJaxenKLLahaPixe37FFUyd2SE4uO5Jk27DZw9m942nMaP4bK1mzV0bLQGfuIngJkZqq8zPx/c+wsXojcPT5I7YOYM8H1Qiso58I5ZSTcSMSNnOLRW68Bfw5/buXPh/IqxsaB+v71bWJzPoagwyipFLA0qRQv5FwL4V+P5NwD8J/sgpdR+APsBYPv27YUOyHS0mc9NivrC2z+0s2fDwpGxd3GKwjfWuGswa9E0GrQrk7lxtj1B+UIwuQqm1lTS4Q1voG3/3vGOdDtkcTvf/z49j5sY7Hu5ukp9uzYSZ+6+m0o6bN/urq/jI03ugH3sLbcATz4ZOFB37qRaQrZDNSqMFaBJCKC2uOqoUsEkwKUjeCz2Zx8XXlpEGKXE31eDSkTXaK3nAcwDwPj4eIqfX3qSfPGKihu2Y5s/85lgR6Gf+zlgxw7SxpJWTowa68JCUEzs+98nwWBWfDTf39igbfdWVsIRJ0ng/jc2SJjv2kXjT7ND1tIS8KpX0XiYViu8AYqtDbqiT66/PlrIb2wEWq45ubZaQTE3XzazawXIG6AAwX2bmAjvyLW6CtxxB/Anf0L93XNP8HxtDbj9duCf/olet7+TZj9c74fH7dqJy6xdZH8nDhzonDjMbOM8avfY967I+HtZIaTAZcPJ6wFgAsD9xvMDAA5EnVN0nHzS+OOibPdm1Idpb7WdZHHnx0Vy2FE7fL0cobG4SI48+xhXREaUb2FuLojoaLfDdmlzwxA+Z2qKnM1mpIht0zfvh8s/wGPhvtmHMT0dtlHv2UPRShx9ctll9Lzdpv6aTRoPj9mOOY+z09u2dzMyyRwHb55ijsv0H/CGLj6Hqb1BixkhZN4Pc2xR398oH0M3dX/iorR6HdQwaKAkm/yjAH5cKfVjAP4NwJsA/EbBfUaSRGMpMm6YNbS3vS38+tmzgWYF+LXGo0dJA2w2gXvvJROLa6yu0gjr66Q5jo7S85/5GeDhh8PH8AYfZr/2yufJJ4Oqls0mcOkSHXvpEr03OhqeOridV74yOPaRR0iD/cM/DGvlAGmnY2N0P86cCVYc589Tv1qHfRrswwDIH8DVQK+8kswzU1OBBn/bbcEqY2ODNhsH6BrZVMT3wad5Li1R6Qbuk6/d3BjFtJWb92FjgzT8hx4KV/RsNIL/jx4Nr6h8dfLtGkJM3Pc3S1Z0HElWD3n9jiRDNyUuyZ/nA8DrAHwVFGXzrrjje5HxmjQkr8hIAw475Bhortg4PNypTSYNqbPbN2OmTa2RozDs9lyrCnvlMz3tbpcfe/bQMXZ79ms8FnPlwf2Y98Puyw5NtLV8U7vme8SrBlcEDlfSdIVbuuLSbU3dvk7zc+VrNCNlOELGjG+/7LJwPL9rhWlu8WiOI4s2a16DeX+6wRzP8HB0vkS3vy3fSnHQQVkZr1rrzwL4bNH9pCFOY+mFw2higrS5hQXSVo8c6dzdibUUgP5nbZBZX4+OCuHt74Dg7/Cwvz2ANGFzA3JTy240gKeeCteMbzTCEUJ795Jt3+TsWfI/uGAtzFx5sLYPhDcoaTTowZr8vn3haJIPfzisXQOB3Xt0lK5leDissZu7VJnXsb5OWj8QdkibPggei3nfnnySxs/jfelLgWeeCc5n/8H+/eGIJQC4//7wCtO1KfmpU8F5WbVZ9hvcfjudf+ed1GY333FePfBq88gR8iP4HP7d/rbs73U31N2+XwnHa9Uoajno+zKNjYUdslpT36Y5id83HW9aA8eOuZ21Cwsk4PRm1Mutt5Ipgds7fpwEt2lKUIqiQeylP286srYWNu80m8Cf/Rn9f/IkCfj9+yk80eS733VvJmKWcJicpPbMSKNGg4SneT98W/rZjluT9fVAWI6OAlu2AF/6Er3ebNIk9JM/CTzxRPg8c4KIM53wdbz97WFTzVNWsLA9eZhmMbPcAhAIQ/Mz5+9jWkep/d1bXQ1MXRcukPlp7950Tn8bV5kF+7eTx2/L/G7zrmRZf58DEQHkUu/LfFRh05AiHDtxCUbsQJubc5cHMJe4ZkalL1U/7hrYaWm243PCuZJzbAehPT7z2KuvdpuM7DHZ17VnT+BgtU0Vdoauz/RkOlzN12+4gfpzJR25xpokY9VlkvI92FHr+6yinK1R45iZ0fqqq8LlMWzTkMtB7PsepCWr0zfPPtKQZkOdqoOyyhqkfVRByGudv00+SamApF/emRn6MbI935eqH3UNthBhgcrX7pqQzIiRqAll9+6wQNu9O7CtNxp+Ye3qN6rKorkzlVlWodUKXuf29+zpFN52hJP9fqPh9pG4sKNtzHaGh6kvO6KGBYpL0Jj3YmiIzp+airZ12xPZzEynb8bsl++Lyx/RbSXUuLIN3f628vp91ilSxyfkxVzTI+zl9d69tMzkCJXJyWRL2fl54H3vC56/6lXAF79I/7daZA9l08YXvkARDknGc/Ag2ZMPHgSe85zORCNzVyjb5m6O+/x58jHYcOJOq0Xmj7jNTc6dI1MJL8sBMiuYGaPr67Qp+JYtwAc+EIzLjEyZn6dr2ro1PB6tw7Zyk3Yb+KVfov+/+U1geTk+C5nNFHytL3853ePTp+mzHh0F7rsvMOWYeQCu2vBAYOM+dgz41KcCP4DL1g0AH/lI+PnHPgZcfnnYBKZUuPb8wYNBcha33+0et3E+r26iePJsg9upfQVOl+Qv81GmJm/GfhcV32tq2RyFwbVTksQvuzRSV4Eql3YeVUfHjqtvt92x2D5t24y5tzXDXbs6NVXfMpnNPT4zyu7d4egVWzs1sa+JtValgogWu21eOfjMGK64dB53lDnOvCZf8TBXnL7LVOYrEmfnPbBJang4iPKJMvv4ri3qe5yVNG1k6a/o6LgqAjHXRGP+SNl+mXbpmuaLZVcqNIWcKVDtH769JHeFFNpmlSSTll0Nc9cu94RnCyueZLgkLvfLz4eG/CYWO+TONru4HpzEtHNn+HVXiWbXNcVNtC6hyhNm3H30+TFc4Z6uz8D1nbBt5z6buT3uq67qvOfmZ5fVJNNrm3qW/lwT7iAIfJ+QF3PNJqbJodEgE4pS3deQ8R179Ghghmi1yAzCyVAHDgTtcZEvgNq+/HIyUXDavJkOzyGFBw8CDz4YRE64imLxNU9Okpnm0UfDY7zlFor+4E26+VwgCKnkCB8gMMVwir0dBTM6GpgejhwJjllZAT70IbompdwF20y0prF8+ctBaCUA/PEfA9/7XthUs2VL+NwtWzqX+RzGai7Vh4bC931jI4g8iTKn2W2b5jD+bLgMAhAOkVxZoRBQ7pNNeKY5wZf8BNBrrVZgmvnnf6a2+H5yZBWXR8hqkkkTHeOLJkvTRpZoHPOcCxfCCXS1jJ6JwyX5y3xUQZOPm/1dGnsaL715rF0C17dMZxODK0rGjjyxzRAzM35HqqmBm2YLs31XZIatJTca1E/UPduzJ6ztu8o7+B67d3feD9ssZN+jLKUjePVkRwRx+YM0WqX9PTHLMLBD1zYL+cw5SbCjk1wmt2612qT3IOq4JGbJtP35+rYDE/o5eiYOiCYfTVIHjKvELaeYJ41btp2e27Z1aiu2ZqYU8Iu/2DkWM0kGoOfnzwfHKEUarnltCwuBJu5ix45w++vr1M4ddwQVLm3n68YGadNveENQ5pbvo2tVsrFBztVPf9p/n5hGA7juOuAVrwDe/356rdmksgx/+7fBPdI6vFK5775wO2Nj4edLS+TEfuYZ4CUvoQJnfD9tPvlJej+Nk84uMmaWVOA++H6YK6ItW6ILprlYWqK/7XZnToGZTNWtNpv0dxKlgbva8K2EszhG7dWPee3dOJT7FpfkL/NRlRBKH2mLRfmwbcMubcXUzPhhpnG7wjJd6fZ2kbAox6YZv22XAeD3ouLSTc3eXJXYx/sKdbnG1WzSKsEOUeQt76amgjII3KcrZt12RNslE8xCYeb/rnuT5XtjX6upZbJ2b4ZImvZ43/Z/5nG+kgJlxIKn1cCLHOOgOGEhmnw+mFq4nYm4uuoPWbSx7bcubWXfPrKh2oWzuJSBvSLYuRN44IHOvjjjEwhnUQIkctpt0lKBoGAXX6uZ7s/t2AXFXJghh+fOBZoqEGxYvncvrUDOn6fX3vxm4K//urO+/vo6leY1yx1wH+vrVN54ZiZcIuDYsc4xff7z1B+HJrraa7cDX8wddwB/9Eed159FC+aSCrya4cf4OL13+eVu27VZDvq229zlB0ytGSD7u7mCOHOGVoVA77TZtBp4HqWOo8YycHZ4E5fkL/NRdU1ea3eoJUcw5K0tuOzfvkxTU1tUqjNczvYF7N4d2PN9mtcNN4T7tm3/Lps+P7gdO+zzqquCVY9Z+jdq+z7fe6afICpD12zHt5JpNsMJR+Y1+qJa7BVZXBKQa7Xl237QXm34QkV9/qRefD/zZFA07qKAhFAWA5sGOBY5alltnxflpDRLG5gVGqP2I41zHi8udpo8OMzRFV43NxcWSEp17jHqE6ZXXx04Yl0lBdiZa9Zaj3q4zD2mMHY5ln3t2hm5gNY/+qNhk48ZzsnZsS4Bb1YS5VDXKIHKY7XHNjXlPjZukrG/T6ZgzxoKLPQnIuQNsmoMvvNse2tcCWCeFHyRB/ZmFL4JxB6PPTm4jnHZ+VnI2Gn0tsZpX5dPYNnCeHiYBOuuXSTg7YkjSoN3CXmlwnHxvrIAu3d3tt1odK6MOI7dbMN1jP2Z2nZ/e4w+oby46K7n44sqSaqF2ysFO8JGNOR64xPyA2eTz1p1Luo8rqBoVh902W59se/msQsLYVvxpUtBPLhSQUlec4/WZhN45zs7Y+ZdYx4bI/usbY/eti2cRm9vo6cU7QNrX9PLXx5E9rjgWP1Tp8i+zyUCTH7qpyhWX+v4OHnmnnuAF7/YH9n05JPhiplcdpnf5821uZLm6Gg4hvwFL+i8Dju/wIb9Fyy6XTkJbB9+5SuBp58OztU6/D04cSL8PTHt7C7M7wOXJxge7sxXcEXrZC21m3eJ3iTt1b0scCG4JH+Zjyps/5flPFc8eVQbtrbH8e52FIm5gYZpfzZjzllrs5fmdgalmRHLy3kz29M1RtPEYeIy/SQxs0QdZ2ex8mPnTrdZp9kMZwKb+QK2H0Ap0s45Z4DvgWmGMVc9rjIPbMLhe8ablJiZvfYqzZclu7gYLkNgR0CZmb9JNsaIKjhnfmb2WLLEofva6oYk7eXdZ92AaPJEVi9+3Hm8sQTXVXdpGWYbzSZtcM0bTbz1rXTMAw9QpMj3vkfPOV7a1B5ZY2MtD6D/W61AWx0ZAd7znuAYrpvO2mGzCbzlLUEmpF0v3YzoAcKx8wsLFLFhrwZc/PzPh2PZGTNblcd/+nTn+cPDVJPdpeFzmxcuAHffTW3wphp794YjjbSmrN7HHw+iVQCKf7///k6N18xMBei1lZUgoujCBXq+sNCpWY6NBd+DlZXgnturNl5dKAW87nVBX1wvne+TayN0W6N1FZyzNfUzZ9yZz2kzSnn7Q141cMRXNxp2kszWLNmvAgZPk9c6f5s8v5c0E9Buw9Y69+zxj8GOkGEN3na0mlo8Z9Wa2mGz2ekgNp13ZtExPta2E0dtA8gPrm1uxoSzZhxni+dIF59zl6/Nl9XIJZnt412v8aqA7d+2c9a10vA5S333yJfjYK7qXKuBJBq577tlrjCHhtyrwqRtmX2bdn9Xu2kRTb57II7XaLoN3+ommcM2DcQ5V30RNKaQtk0+u3Z1OjztH7WdUMNOUj7W3AuVo1umpgIziE848jhNpyCbSnyCnuvOxzl3uZyCb89PO/opqenINbHYY3CVHrAnYXOitUNf7esyHaVRzlZfcTsbVwimvQFK3HfMTq5ymYXyKHzmGkfWYwYVEfIRJNUi4mKgu4lJtmvQRI3LNRbzWDN0jgWWq0aK+YOMCofk837oh8Kv3XBDoNU1Glrv2EHjn5sjDd4ULvZGGKYmaW5i7RL0c3PhcsX2w9xc3JeVOjen9QtfmEzIc99x/gRXqKxZn8bWcF0hrabWHhfyyMf7Vgc2aaK+7PN8viPXZ2hes2jY5SFCPoI4LTyNKSZuqR2FLbzNH6kvEca8BlOQs0Zob/G3a5d7fEnCIZMKwpmZztXJDTd0Fsji++UT8nwtnNQVlcBkvsbOVLtsRFTbtmOXd3Oyr9c8lydlux+O/+eViKtks3nfzdWQ73tmb9foWh24vk9JyiL4zvMpBLZZj1dJfM1FIpq8HxHyEcQJ8awVJuOW0lFmGFeER9QPyCVU9+xxZ6xG2VtdiUJR2r3LBMI+AFtwm+aUxcVk0Tm2kLU1zJkZtzA2JzLTzOSaJHbt6rxus+ok92tPMvaGIlEbg/hi+e0oGlf9o8XFzg1BouLwo75nSeEJ2Lf1YdxEkNc47DbEJu/HJ+QHLrrGRVydjaQROUnrhLji112RA0AQe91oBDHyLlZWwvH0H/1ocD6/zm34annYseVxtFoUn29uRwhQX5/+dFDbx3z94kWqVf/c5yaLzmG43s0b30hRM9/5DrX3/vfTe3b/Fy8Gf4FwDR6T9XXgkUco2shk2za6R7OzFCnznOfQFnx2P0AQYQJQO+vr9N6xY0F9e9d3iD9z5tIld/2jhYXw2JtN4NZbw7XzfcTVbfHFnfN5+/ZF14TnexC198LSEr1+6RLVBsoaFSPRNRlxSf4yH1WNrsnTJu/T6lz2anvnItdYzLIHLq06KuvRXHqbdnRA6+c9L1oD5pWFnR3K2u3UVKcGmuXBKwO7PILvWJctnJ28UasPfp3t+uZnElex0qx+6XOKxn1uPvu6ywyUB91oxmm+73ZmcNJa+UlWu0IABlmTN7UVIFvGa5xGZGcoAoE2bmcamlUclaKYdlNr3LkzqIXN7a2tkaZtj9+ulGjTbAL33uveUcjOkrR5y1toTHZ9daVIk+TcgMlJ0oZNhocp3v/664G3v70zTt5s6+qrKRY+ivvvp02143jpS2mHJSBcT5yvUalAa7evmWPWtabVyTe/Ga4CabKxAUxN0fXZ99a3+5JdX/7OO6mvZpPq8M/MuL9jrt2h5uf9u0SZRGWIptWM7baK3AA7bqN3yXhNgUvyl/nIW5O3Z/+8wr3sPqKyVF1ONzsiwdytibVD29HWbofH73J8srZn2qx9REXU3HADHWPXujFDIs2qj+YxXGWS+4gKXWQt3aXxmyGILger6zE11em8tq+x2Uzne3B9FgCtYHzfh7iVYpaQW1Ojt1cQUcenjbfP0lbcuF0r0ih890ccr34wqJq8ra0A4T08z53Lpw9zB6exsaA2ysWL/j1WWUM0jzHr32gdzgzlPngvVZf2bdqntaY67PbepwyvKOwVCAD84z/S/q4/+IPBDlXtNnDTTXR9rB03m8Cv/3rYFv31rwPveAfwuc91jslGa9rBibVsexytFq1iGg2yk8/MUGbszp3AV79K/z/7bHDsffdRFuvQEO3faq6a2LcxNEQZvKdOdfbnYnyc+rGPXVkhjdNVAyZuX4Ekfh5bc+bvsvkdcWngURmutt09qWbcjT18YsK9l24UrvuTte7UwOOS/GU+itbkFxc7bbvd2jjtPmzN3aXJ+86xtV5OIDI1/ig7ty8UMapmuUtDbjbDiTymLdilnbNN2xfvbkdhmG2b/8eNg2vGmPeKx+fLSDXrApnhjradnVcL5uv26sk+nqt+ckZxnuWmfd8ZnyZvR8SYGbeuDOc0lGEPd4UU570K73ZMVQKDHEJpfzBpyghk7SPuue8Y1wbZ5o/Uta2d+Wi3/eUGojYbMQXs857nFmj8o7LrzJtj9Zl/WIBOT3duJM6x6kND4QnMN2G5EqNc2/WxuSoqPp0nEXMSY4eoWXjMLlVgJ5m5Jlu7Br3p5M5qzvG14QpJte8JT1hZBVTZAq5sx2vZ/cfRcyEP4NcAfAXABoDxpOdlFfJpvoC+RJ0iPjTzRxlX6z0udnxqqlPr3ro1rNGyEHUJ26uv7ozaabf91R/NR6sV9O8aIwvKqPID5qQyMxMczzZybv+nfzo6KYvHYGr4psbvE3Su+8Nx8uaOV3b5Bf7c+Fyz77gHr6BsLdxVgsH+3qTJoHZN/r7PI0lVS9f3swqUOaYqrCSiKEPIXw3gJQAWihbyWWZYu3hVESnZ9g/b/OG7TDhxpQVMp6qpndkJK1Ht8IYgSQSU3XfUe1FlB8x+eaxJd4RyPWZmgknKvA+u+jR2HRqzdIDvM7FLQpghqHEF01yT36FD/s8kLmM1aQa1/ZlyiYkkq7qo725VtdYyqPo98Qn5RoG2/qe11s8U1b6JL5EoissvDzsETSdWFpaWyFG5tNQ5LttB6nPGsrOJN9l2JflwCVpmfZ1K0d51V+CI4naazc4kn0uXkoUimrRa1LcL3pzimmvcx7TbFGqoFDlE+ZrZ0ZqWRoOcoGtr4f42Nsgp/N73knO21aJjt2yh8ETui/u2uXCBQi/58zKPv3SJ/nKZYfMzcmFeFzsN+TOxr/nsWfd3B6DPcvt2uta477a5ATsA7N5NIa633OIfJ+DvO+lvynd+HWFHtflby4tC76NL8uf5QAJNHsB+AMsAlrdv3556Bssyw6YNR8vSv8/84tPk+RzT5mqW6XVpqs2m33HH7bhS510mh0aDzCasZbdagYnCbINtu766LWx+YRODr+CVabJJqhX7wk1daff22GxnpP3YvTs4zqetm2UhXKWMgcD3YJtYXD4Xc9vFbsIc+btmm4FcZiLbhBRVtiCq36prtv1CXvcRRYRQKqU+D2Cb4613aa0/mWKimQcwDwDj4+MendFPliQJV4JJ3hseTEyQlj03F2idu3ZRgtHEBG1uYY/ZlXQ1NRUc9773hbfme8Mb/Ak0/ProaLAJBodSct8jIxQKaL7nSqBxtWET9RlwOFyrRRo3t8HXZo7j7FnS+vmeKUUrAt5kxR6/67Oz76M5NoCu5eGHw0lYO3YAf/AHdNwjj3RugegqC/HiF4eTvdrt6KSm2dlwiv+2bfmEOXKIpev75PueHz7s7ztJv1JmIB8Kv48uyZ/nAz2wyZdNtxpR2r58tdOrTBqHmXnP7HrmeY/Jl6Rjvhe30mPbedJx+lYZvdaGu+1bNPl8KFqTV9pnbM0JpdQCgP+itV5Ocvz4+LheXk50aKWISh8vY8PjfqdX15jkc+t2pdfNGIqm274H4bvYC/K4j0qpx7TW4x2vFyXklVK/CuAeAFsBnANwWmv9i3Hn9auQFwRBKBOfkC+srIHW+uMAPl5U+4IgCEI8hYVQCoIgCOUjQl4QBKHGiJAXBEGoMSLkBUEQaowIeUEQhBpTeJx8WpRS3wbwbNnj2OT5AL5T9iByQq6letTlOgC5lirwo1rrrfaLlRPyVUIpteyKO+1H5FqqR12uA5BrqTJirhEEQagxIuQFQRBqjAj5aObLHkCOyLVUj7pcByDXUlnEJi8IglBjRJMXBEGoMSLkBUEQaowI+RiUUu9XSv2DUurLSqmPK6UuL3tMWVFK/ZpS6itKqQ2lVN+FiCmlrlNKPaOU+ppS6vfKHk9WlFJHlVLfUkr9fdlj6Ral1I8opR5SSj21+d367bLHlAWl1Bal1CNKqSc2r+M9ZY8pL0TIx/MggJdprX8KwFcBHCh5PN3w9wDeCODhsgeSFqVUE8AHAVwPYAeANyuldpQ7qsz8BYDryh5ETqwB+B2t9Q4ArwBwW59+LhcAvFpr/dMAdgK4Tin1inKHlA8i5GPQWj+gtV7bfPolAC8qczzdoLV+Wmv9TNnjyMguAF/TWn9da30RwF8C+JWSx5QJrfXDAL5b9jjyQGv9v7XWj2/+/38APA3gheWOKj2bO+j9++bT9uajFlEpIuTTcTOAz5U9iAHlhQD+1Xj+DfShMKkzSqkrAIwB+LuSh5IJpVRTKXUawLcAPKi17svrsClsZ6h+Qin1eQDbHG+9S2v9yc1j3gVamn6kl2NLS5JrEYS8UUr9AICTAO7UWn+v7PFkQWu9DmDnpt/t40qpl2mt+95vIkIegNb6tVHvK6V+C8DrAbxGVzyxIO5a+ph/A/AjxvMXbb4mlIxSqg0S8B/RWn+s7PF0i9b6nFLqIZDfpO+FvJhrYlBKXQdgBsAva63/b9njGWAeBfDjSqkfU0oNAXgTgE+VPKaBRymlAHwYwNNa67vLHk9WlFJbOXJOKXUZgF8A8A+lDionRMjHcy+A5wJ4UCl1Win1P8oeUFaUUr+qlPoGgAkA9yml7i97TEnZdH7fDuB+kHPvr7TWXyl3VNlQSn0UwBKAlyilvqGUuqXsMXXBzwL4TQCv3vx9nFZKva7sQWXghwE8pJT6MkiheFBr/ZmSx5QLUtZAEAShxogmLwiCUGNEyAuCINQYEfKCIAg1RoS8IAhCjREhLwiCUGNEyAuCINQYEfKCIAg1RoS8IMSglHr55n4CW5RS/2Gz3vjLyh6XICRBkqEEIQFKqfcC2ALgMgDf0FofLnlIgpAIEfKCkIDNejmPAjgP4Gc2KxYKQuURc40gJGMEwA+A6hhtKXksgpAY0eQFIQFKqU+BdqP6MQA/rLW+veQhCUIipJ68IMSglNoH4JLW+n9t7jW7qJR6tdb6i2WPTRDiEE1eEAShxohNXhAEocaIkBcEQagxIuQFQRBqjAh5QRCEGiNCXhAEocaIkBcEQagxIuQFQRBqzP8DT1Tz7WNGo+8AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "lr = 10e-7\n",
        "n_iter = 200\n",
        "\n",
        "cost_history_list = []\n",
        "\n",
        "# batch size\n",
        "batch_size_list = [1, 10, 100, X_df.shape[0]]\n",
        "\n",
        "for bt_size in batch_size_list:\n",
        "  \"\"\"\n",
        "  When,\n",
        "    bt_sze = 1 -> Stocastic gradient\n",
        "    bt_sze = m (100) -> Batch gradient\n",
        "  \"\"\"\n",
        "  y_pred = np.random.randn(2, 1)\n",
        "  y_pred, cost_history = minibatch_gradient_descent(X_df, y_df, y_pred, learning_rate=lr, iterations=n_iter, batch_size=bt_size)\n",
        "\n",
        "  cost_history_list.append(cost_history)\n",
        "\n",
        "  if bt_size == 1:\n",
        "    gd_name = \"Stocastic Gradient Descent\"\n",
        "  elif bt_size == batch_size_list[-1]:\n",
        "    gd_name = \"Batch Gradient Descent\"\n",
        "  else:\n",
        "    gd_name = \"Mini-Batch Gradient Descent\"\n",
        "\n",
        "  print(f\"{gd_name}\")\n",
        "  print(f\"Batch Size = {bt_size}\")\n",
        "  print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "  print('Final error: {:0.3f}'.format(cost_history[-1]))\n",
        "  print(\"\")"
      ],
      "metadata": {
        "id": "cszUsjpCwLgH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "fig, ax = plt.subplots(figsize=(10,8))\n",
        "\n",
        "ax.set_ylabel('Error')\n",
        "ax.set_xlabel('Number of iterations')\n",
        "y_pred = np.random.randn(2,1)\n",
        "\n",
        "for bt_size, cost_history in zip(batch_size_list, cost_history_list):\n",
        "  ax.plot(range(n_iter), cost_history, label=str(bt_size))\n",
        "\n",
        "plt.ylim([0, 10e6])\n",
        "\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "7LlqNAlywVSG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#**ANSWER FOR 3RD QUESTIION**"
      ],
      "metadata": {
        "id": "7s2hDIDbwkO2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lr_rates = [0.1, 0.01, 0.0001, 10e-4, 10e-5]\n",
        "n_iter = 1000\n",
        "\n",
        "cost_history_all = []\n",
        "\n",
        "for lr in lr_rates:\n",
        "\n",
        "  d = {}\n",
        "\n",
        "  print(f\"Learning Rate: {lr}\")\n",
        "  # Batch \n",
        "  y_pred = np.random.randn(2,1)\n",
        "  X_b = np.c_[np.ones((len(X), 1)), X]\n",
        "  y_pred, cost_history, y_pred_history = gradient_descent(X_b, y, y_pred, lr, n_iter)\n",
        "  d[\"bgd\"] = cost_history\n",
        "\n",
        "  print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "  print('Batch GD: Final error: {:0.3f}'.format(cost_history[-1]))\n",
        "\n",
        "  # Stocastic\n",
        "  y_pred = np.random.randn(2, 1)\n",
        "  X_b = np.c_[np.ones((len(X), 1)), X]\n",
        "  y_pred, cost_history = stocashtic_gradient_descent(X_b, y, y_pred, lr, n_iter)\n",
        "  d[\"sgd\"] = cost_history\n",
        "\n",
        "  print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "  print('SGD: Final error: {:0.3f}'.format(cost_history[-1]))\n",
        "\n",
        "\n",
        "  # Mini Batch\n",
        "  y_pred = np.random.randn(2,1)\n",
        "  y_pred, cost_history = minibatch_gradient_descent(X, y, y_pred, lr, n_iter)\n",
        "  d[\"mgd\"] = cost_history\n",
        "\n",
        "  print('y_pred[0]: {:0.3f}\\ny_pred[1]: {:0.3f}'.format(y_pred[0][0], y_pred[1][0]))\n",
        "  print('Mini-Batch GD Final error: {:0.3f}'.format(cost_history[-1]))\n",
        "\n",
        "  cost_history_all.append(d)\n",
        "  print(\" \")"
      ],
      "metadata": {
        "id": "Tr0QVeJ7wg8k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "for lr, cost_history in zip(lr_rates, cost_history_all):\n",
        "  fig, ax = plt.subplots(figsize=(10,8))\n",
        "\n",
        "  ax.set_title(f\"Learning rate is {lr}\")\n",
        "  ax.set_ylabel('Error')\n",
        "  ax.set_xlabel('Number of iterations')\n",
        "  y_pred = np.random.randn(2,1)\n",
        "\n",
        "  ax.plot(range(n_iter), cost_history[\"bgd\"], label=\"Batch\", linestyle=\"solid\")\n",
        "  ax.plot(range(n_iter), cost_history[\"sgd\"], label=\"Stocastic\", linestyle=\"dotted\")\n",
        "  ax.plot(range(n_iter), cost_history[\"mgd\"], label=\"Mini-Batch\", linestyle=\"dashed\")\n",
        "\n",
        "  # plt.ylim([0, 10e6])\n",
        "\n",
        "  plt.legend()\n",
        "  plt.show()"
      ],
      "metadata": {
        "id": "vbE3cP5ewvyB"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}